-- Logs begin at Mon 2022-05-09 22:11:50 UTC, end at Mon 2022-05-09 23:27:48 UTC. -- May 09 22:47:40.981421 dv-cinder-840678-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 09 22:47:40.989033 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 09 22:47:40.991087 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon May 9 22:47:40 2022] *** May 09 22:47:40.991087 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 09 22:47:40.991087 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 09 22:47:40.991087 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: nodename: dv-cinder-840678-5-iscsi May 09 22:47:40.991087 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: machine: x86_64 May 09 22:47:40.991087 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: clock source: unix May 09 22:47:40.991087 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: pcre jit disabled May 09 22:47:40.991087 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: detected number of CPU cores: 4 May 09 22:47:40.991601 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: current working directory: / May 09 22:47:40.991601 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: detected binary path: /usr/bin/uwsgi-core May 09 22:47:40.991601 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: your processes number limit is 47680 May 09 22:47:40.991601 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: your memory page size is 4096 bytes May 09 22:47:40.991601 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: detected max file descriptor number: 2048 May 09 22:47:40.991601 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: lock engine: pthread robust mutexes May 09 22:47:40.991601 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: thunder lock: enabled May 09 22:47:40.991601 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 09 22:47:40.991601 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 09 22:47:41.038499 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: Python main interpreter initialized at 0x5611f99dd9c0 May 09 22:47:41.038499 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: python threads support enabled May 09 22:47:41.038499 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: your server socket listen backlog is limited to 100 connections May 09 22:47:41.038499 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: your mercy for graceful operations on workers is 90 seconds May 09 22:47:41.038845 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: mapped 403077 bytes (393 KB) for 2 cores May 09 22:47:41.038963 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: *** Operational MODE: preforking *** May 09 22:47:41.039247 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: *** uWSGI is running in multiple interpreter mode *** May 09 22:47:41.039247 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: spawned uWSGI master process (pid: 79877) May 09 22:47:41.039447 dv-cinder-840678-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 09 22:47:41.040024 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: spawned uWSGI worker 1 (pid: 79887, cores: 1) May 09 22:47:41.040387 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: spawned uWSGI worker 2 (pid: 79888, cores: 1) May 09 22:47:41.040655 dv-cinder-840678-5-iscsi devstack@keystone.service[79877]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 09 22:47:46.770872 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 22:47:46.770872 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: from cryptography.utils import int_from_bytes May 09 22:47:46.772133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 22:47:46.772133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: from cryptography.utils import int_from_bytes May 09 22:47:46.774692 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 22:47:46.774692 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: from cryptography.utils import int_from_bytes May 09 22:47:46.775979 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 22:47:46.775979 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: from cryptography.utils import int_from_bytes May 09 22:47:46.822783 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 09 22:47:46.822959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 09 22:47:46.823151 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] command line args: [] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 09 22:47:46.823425 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 09 22:47:46.823608 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ================================================================================ {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 09 22:47:46.823866 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] admin_token = **** {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.824086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 09 22:47:46.824169 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] config_dir = [] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.824305 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 09 22:47:46.824391 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.824479 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] config_source = [] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.824549 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] command line args: [] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 09 22:47:46.824615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] debug = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.824806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 09 22:47:46.824895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.825118 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ================================================================================ {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 09 22:47:46.826104 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] admin_token = **** {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.826249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] config_dir = [] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.826311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.826501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.826597 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] insecure_debug = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.826666 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] config_source = [] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.826736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.826796 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] debug = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.827159 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.827352 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.827575 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] list_limit = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.827792 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.827983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] log_config_append = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.828176 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] insecure_debug = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.828360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.828554 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.828777 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] log_dir = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.828939 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.829132 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] log_file = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.829325 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] list_limit = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.829531 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.829755 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] log_config_append = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.829925 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.830148 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.830148 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] log_dir = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.830484 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.830664 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] log_file = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.830885 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.831087 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.831250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.831250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.831571 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.831712 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.831875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.832012 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.832179 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.832179 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.832432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.832557 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.832685 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.832817 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.832984 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.833109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.833235 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.833369 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.833510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.833651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.833777 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] notification_format = cadf {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.833901 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.834034 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.834218 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.834218 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.834475 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.834601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] notification_format = cadf {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.834732 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] publish_errors = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.834866 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.834998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.834998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.834998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.835447 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.835447 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] publish_errors = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.835703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.835835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.835966 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.836097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.836281 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] standard_threads = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.836417 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.836543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] strict_password_check = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.836667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.836793 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.837016 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.837155 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] use_eventlog = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.837280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] standard_threads = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.837405 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] use_journal = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.837405 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] use_json = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.837640 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] strict_password_check = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.837839 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] use_stderr = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.837971 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.838097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] use_syslog = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.838265 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] use_eventlog = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.838391 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] watch_log_file = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.838516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] use_journal = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.838516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] use_json = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.838761 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.838893 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] use_stderr = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.839035 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.839166 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] use_syslog = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.839291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.839423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] watch_log_file = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 22:47:46.839553 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.839678 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.839802 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.839927 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.840059 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.840189 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.840321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.840446 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.840569 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.840703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.840909 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.841043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.841174 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.841401 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.841561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.841691 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.841824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.841955 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.842082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.842263 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.842391 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.842516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.842653 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.842790 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.842921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.843048 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.843175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.843305 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.843442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.843576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.843710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.843836 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.843960 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.844084 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.844221 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.844347 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.844481 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.844606 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.844606 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.844841 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.844973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.845098 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.845229 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.845354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.845354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.845590 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.845723 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.845853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.845985 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.846147 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.846282 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.846476 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.846633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.846763 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.846763 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.846999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.847123 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.847248 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.847452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.847587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.847713 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.847838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.847963 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.848103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.848237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.848364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.848488 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.848612 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.848742 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.848876 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.849002 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.849126 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.849257 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.849389 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.849515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.849648 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.849784 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.849909 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.850034 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.850198 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.850337 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.850472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.850598 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.850723 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.850848 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.850972 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.851103 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.851236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.851361 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.851493 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.851865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.852006 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.852139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.852274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.853108 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.853343 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.853512 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.853701 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.853889 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.854119 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.854296 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.854549 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.854746 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.854921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.855110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.855338 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.855508 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.855699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.855887 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.856075 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.856264 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.856490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.856654 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.856843 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.857158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.857321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.857517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.857753 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.857954 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.858152 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.858325 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.858499 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.858632 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.858758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.858931 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.859069 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.859205 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.859332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.859458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.859616 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.859759 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.860145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.860278 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.860410 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.860536 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.860661 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.860793 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.860960 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.861094 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.861225 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.861352 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.861478 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.861478 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.861478 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.enabled = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.861478 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.861969 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.862095 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.862242 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.862368 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.862504 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.862637 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.862763 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.862888 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.863014 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.enabled = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.863145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.863345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.863486 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.863668 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.863830 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.863957 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.864092 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.864219 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.864345 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.864470 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.864596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.864727 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.864862 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.864989 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.865115 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.865241 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.865367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.865502 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.865635 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.865762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.865894 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.866020 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.866187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.866320 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.866457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.866595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.866733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.866865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.866993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.867118 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.867244 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.867385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.867521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.867655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.867782 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.867908 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.868034 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.868212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.868848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.868976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.869108 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.869235 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.869361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.869486 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.869630 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.869763 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.869889 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] auth.external = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.870015 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.870156 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] auth.mapped = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.870298 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.870434 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.870568 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.870696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.870822 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.870948 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] auth.password = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.871081 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.871235 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] auth.token = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.871375 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] auth.external = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.871522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.871650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] auth.mapped = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.871777 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] catalog.caching = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.871904 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.872032 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.872166 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.872293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.872426 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] auth.password = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.872555 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.872682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] auth.token = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.872809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.872944 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.873071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.873208 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] catalog.caching = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.873335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] credential.caching = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.873467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.873599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] credential.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.873734 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.873861 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.873988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.874157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.874293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.874420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.874560 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.874688 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.874815 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] credential.caching = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.874942 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.875078 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] credential.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.875205 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.875340 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.875468 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.875632 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.875769 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.875897 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.876033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.876169 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.876169 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.876464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.876592 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.876719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.876856 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.876991 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.877119 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.877245 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.877371 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.877498 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.877498 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] federation.caching = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.877747 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.877874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.878001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.878145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] federation.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.878279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.878406 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.878591 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.878736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.878864 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.878990 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.879117 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] federation.caching = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.879258 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.879460 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.879460 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] federation.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.879701 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.879830 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.879955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.880081 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.880215 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.880343 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.880483 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.880610 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.880736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.880864 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.880999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.caching = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.881127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.881253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.881380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.881512 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.881640 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.881774 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.881903 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.882029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.882172 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.caching = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.882315 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.882442 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.882586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.882714 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.882842 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.882969 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.883096 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.883223 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.883358 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.883495 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.883622 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.883770 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.883903 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.884051 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.884178 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.884313 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.884441 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.884567 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.884698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.884837 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.884973 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.885101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.885227 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.885354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.885480 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.885615 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.885742 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.885882 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.886009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.886185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.886320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.886458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.886586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.886713 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.886840 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.886974 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.887101 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.887237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.887365 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.887492 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.887629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.887756 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.887881 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.888016 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.888150 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.888280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.888406 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.888532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.888659 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.888793 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.888921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.889047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.889180 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.889309 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.889445 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.889580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.889706 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.889833 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.889959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.890086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.890237 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.890377 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.890504 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.password = **** {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.890631 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.890757 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.890890 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.891018 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.891152 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.891289 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.891417 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.891543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.891669 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.password = **** {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.891795 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.891929 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.892057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.892191 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.892318 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.892446 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.892572 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.892711 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.892839 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.892965 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.893102 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.893229 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.893361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.893501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.893630 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.893756 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.893882 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.894009 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.894176 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.894322 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.894450 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.894578 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.894705 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.894842 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.894842 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.895090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.895218 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.895346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.895473 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.895600 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.895727 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.895862 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.895989 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.896117 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.896243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.896370 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.896497 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.896641 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.896776 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.896904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.897030 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.897157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.897291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.897418 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.897545 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.897672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.897799 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.897925 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.898058 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.898208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.898346 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.898473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.898600 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.898728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.898865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.898993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.899120 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.899246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.899394 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.899527 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.899661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.899790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.899916 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.900042 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.900178 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.900311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.900446 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.900574 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.900706 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.900835 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.900961 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.901088 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.901221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.901349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.901475 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.901602 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] policy.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.901729 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.901857 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.902000 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.902176 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.902311 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.902438 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.902563 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] policy.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.902690 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] receipt.caching = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.902827 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.902956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.903082 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.903208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.903340 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.903471 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.903605 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] receipt.caching = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.903742 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.903868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.903996 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.904122 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.904248 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] resource.caching = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.904449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.904579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.904705 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.904832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] resource.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.904960 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.905087 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.905221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] resource.caching = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.905349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.905476 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.905612 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.905739 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] resource.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.905865 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] revoke.caching = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.906004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.906149 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.906282 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.906409 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.906545 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.906674 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] role.cache_time = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.906812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] revoke.caching = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.906940 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] role.caching = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.907069 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.907196 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] role.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.907323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.907460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] role.list_limit = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.907600 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] role.cache_time = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.907729 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.907855 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] role.caching = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.907982 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.908111 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] role.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.908238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.908373 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] role.list_limit = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.908501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.908629 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.908629 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.908869 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.908996 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.909145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.909274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.909401 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.909528 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.909655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.909782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.909917 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.910045 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.910210 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.910340 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.910467 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.910606 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.910734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.910861 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.910998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.911125 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.911253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.911389 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.911516 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.911643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.911769 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.911897 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.912025 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.912160 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.912288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.912415 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.912542 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.912679 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.912806 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.912941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.913068 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.913195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.913329 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.913462 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.913589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.913724 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.913852 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.913979 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.914270 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.914435 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.914577 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.914717 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.914845 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.915498 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.915634 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.915762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.915889 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.916024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.916153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.916280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.916407 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.916535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.916662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] token.cache_time = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.916808 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.916936 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] token.caching = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.917062 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.917189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.917316 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] token.cache_time = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.917451 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] token.provider = fernet {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.917579 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] token.caching = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.917705 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.917837 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.917963 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.918090 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] token.provider = fernet {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.918252 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.918382 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.918512 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.918650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.918778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.918905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.919039 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.919167 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.919294 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] trust.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.919421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.919547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.919675 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.919810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.919984 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] trust.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.920111 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.920238 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.920364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.920501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.920636 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.920771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.920898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.921026 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.921246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.921415 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.921560 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.921695 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.921695 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.921950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.connection = **** {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.921950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.922275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.922423 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.922563 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.connection = **** {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.922710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.922858 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.922992 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.923126 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.923266 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.923404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.923537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.923679 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.923815 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.923950 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.924083 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.924217 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.924360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.924503 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.924637 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.924771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.924906 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.925040 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.925174 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.925316 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.925451 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.925584 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.925717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.925850 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.925985 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.926215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.926370 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.926503 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.926637 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.926771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.926771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.927034 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.927034 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 09 22:47:46.927290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.927290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 22:47:46.927290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 09 22:47:46.941337 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:46.941649 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:46.941820 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:46.941976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:46.942139 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:46.942287 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:46.950506 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:46.950676 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:46.982476 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:46.982733 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:46.983508 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:46.983742 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.038765 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.039571 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.092924 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.093163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.097749 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.097986 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.159291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.159471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.159563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.159687 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.159789 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.159907 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.159991 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.160124 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.160207 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.160341 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.160425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.160557 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.184036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.184302 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.184549 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.184771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.184857 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.184987 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.185069 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.185202 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.185294 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.185417 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.185500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.185627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.185838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.186049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.196801 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.197465 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.211319 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.211525 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.211635 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.211738 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.212371 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.212525 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.212609 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.212748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.212832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.212963 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.213047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.213177 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.213259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.213391 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 22:47:47.233118 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.233325 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 22:47:47.233662 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.233748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.233850 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 22:47:47.233930 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 22:47:47.234151 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.234241 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.234393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.234477 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 22:47:47.234647 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.234733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.235145 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.235233 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.235437 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.235690 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.246152 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.246427 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.246689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.246774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.246962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.247137 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.247234 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.247494 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.255038 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.255555 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.258376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.258578 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.258862 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.258991 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.259140 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.259272 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.259530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.259613 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.259797 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.259878 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.260062 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.260153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.260327 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.260409 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.260593 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.260673 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.260858 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.260941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.261118 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.261199 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.261385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.261648 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.288099 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.288639 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.289712 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.290623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.290837 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.291111 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.295912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.296189 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.296463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.296693 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.296973 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.297246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.303765 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.304042 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.304570 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.304846 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.308651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.308933 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.309211 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.309484 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.309577 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.309748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.309827 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.310013 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.310091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.310302 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.310395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.310565 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.310645 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.310832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.311101 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.311366 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.329506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.329776 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.330382 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.330657 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.334290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.335127 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.338496 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.338780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.339288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.339577 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.343698 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.344488 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.347261 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.348039 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.350926 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.351220 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.351502 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.351741 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.351823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.352035 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.352195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.352306 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.352386 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.352612 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.352885 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.353159 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.371551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.371920 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.372186 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.372311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.372449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.372643 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.372911 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.373173 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.380267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.381035 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.381966 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.382262 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.382775 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.383047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.385834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.386107 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.386397 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.386627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.386754 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.386902 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.387040 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.387166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.387451 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.387723 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.400833 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.401106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.401380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.401665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.401871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.402006 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.402168 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.402293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.402418 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.402543 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.402706 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.402992 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.403277 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.403549 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.420570 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.421480 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.423805 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.424750 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.427095 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.428063 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.428672 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.429648 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.430314 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.430616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.430903 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.431342 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.431618 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.431908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.438334 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.439273 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.439919 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.440858 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.443137 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.443422 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.443700 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.443973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.444098 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.444340 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.444617 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.444886 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.453652 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.453940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.454239 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.454618 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.454909 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.455184 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.462823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.463095 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.463369 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.463637 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.463760 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 22:47:47.463887 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.464011 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.464165 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.464304 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.464430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.464554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.464688 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.464811 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.464951 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.465076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.465200 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.465333 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.465513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.465639 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.465764 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.465887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.466020 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.466200 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.466328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.466452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.466576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.466738 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.467003 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 22:47:47.504742 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5611f99dd9c0 pid: 79887 (default app) May 09 22:47:47.505888 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5611f99dd9c0 pid: 79888 (default app) May 09 22:47:47.508096 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e9b08-e371-4129-aeb7-64362a52ddea None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:47:47.508388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e9b08-e371-4129-aeb7-64362a52ddea None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:47:47.509184 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e9b08-e371-4129-aeb7-64362a52ddea None None] PATH_INFO: `/v3` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:47:47.513420 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Mon May 9 22:47:47 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:47:55.495629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a909011-94e0-4613-9dc9-659fa6b7661a None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:47:55.496200 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a909011-94e0-4613-9dc9-659fa6b7661a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:47:55.496200 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a909011-94e0-4613-9dc9-659fa6b7661a None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:47:55.500534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1/2] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:47:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:47:55.505588 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699f73e-36ed-41ad-a8b5-8b5891441f81 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:47:55.506021 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699f73e-36ed-41ad-a8b5-8b5891441f81 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:47:55.506170 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699f73e-36ed-41ad-a8b5-8b5891441f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:47:55.536878 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG oslo_db.sqlalchemy.engines [None req-5699f73e-36ed-41ad-a8b5-8b5891441f81 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=79887) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 09 22:47:55.728518 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG passlib.handlers.bcrypt [None req-5699f73e-36ed-41ad-a8b5-8b5891441f81 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=79887) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 09 22:47:55.728943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG passlib.handlers.bcrypt [None req-5699f73e-36ed-41ad-a8b5-8b5891441f81 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=79887) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 09 22:47:55.783766 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-5699f73e-36ed-41ad-a8b5-8b5891441f81 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:47:55.785391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5699f73e-36ed-41ad-a8b5-8b5891441f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:47:55.906595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 2/3] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:47:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 09 22:47:56.043657 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a909011-94e0-4613-9dc9-659fa6b7661a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:47:56.076538 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG oslo_db.sqlalchemy.engines [None req-2a909011-94e0-4613-9dc9-659fa6b7661a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=79888) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 09 22:47:56.202345 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a909011-94e0-4613-9dc9-659fa6b7661a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:47:56.206574 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04367a7-3daf-4e98-b871-ec846fe25929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:47:56.207226 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04367a7-3daf-4e98-b871-ec846fe25929 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:47:56.207511 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04367a7-3daf-4e98-b871-ec846fe25929 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:47:56.207790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04367a7-3daf-4e98-b871-ec846fe25929 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:47:56.216256 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04367a7-3daf-4e98-b871-ec846fe25929 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:47:56.267889 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e04367a7-3daf-4e98-b871-ec846fe25929 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 09 22:47:56.268231 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04367a7-3daf-4e98-b871-ec846fe25929 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:47:56.278886 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 2/4] 10.222.0.23 () {58 vars in 1324 bytes} [Mon May 9 22:47:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 22:47:56.284773 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699f73e-36ed-41ad-a8b5-8b5891441f81 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:47:56.299309 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699f73e-36ed-41ad-a8b5-8b5891441f81 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:47:56.303737 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad7a74-21dd-4376-8707-b709e73bc9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:47:56.304898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad7a74-21dd-4376-8707-b709e73bc9bf admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:47:56.305245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad7a74-21dd-4376-8707-b709e73bc9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:47:56.305581 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad7a74-21dd-4376-8707-b709e73bc9bf admin admin] PATH_INFO: `/v3/projects/4202d89a17374bc0a3d992386d254d4c` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:47:56.307187 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad7a74-21dd-4376-8707-b709e73bc9bf admin admin] RBAC: Authorizing `identity:get_project(project_id=4202d89a17374bc0a3d992386d254d4c)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:47:56.357082 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: WARNING keystone.common.rbac_enforcer.enforcer [None req-bbad7a74-21dd-4376-8707-b709e73bc9bf admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 09 22:47:56.357420 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad7a74-21dd-4376-8707-b709e73bc9bf admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:47:56.362209 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 3/5] 10.222.0.23 () {56 vars in 1207 bytes} [Mon May 9 22:47:56 2022] GET /identity/v3/projects/4202d89a17374bc0a3d992386d254d4c => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:47:58.552427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e651bf8-5fc9-4783-83fc-8fc42d180c1b None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:47:58.552974 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e651bf8-5fc9-4783-83fc-8fc42d180c1b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:47:58.552974 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e651bf8-5fc9-4783-83fc-8fc42d180c1b None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:47:58.557662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 3/6] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:47:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:47:58.563030 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc074d4e-c19a-4ed0-97e2-898a724b560a None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:47:58.563397 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc074d4e-c19a-4ed0-97e2-898a724b560a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:47:58.563590 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc074d4e-c19a-4ed0-97e2-898a724b560a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:47:58.583833 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-dc074d4e-c19a-4ed0-97e2-898a724b560a None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:47:58.585351 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-dc074d4e-c19a-4ed0-97e2-898a724b560a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:47:58.593096 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 4/7] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:47:58 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 09 22:47:58.615663 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e651bf8-5fc9-4783-83fc-8fc42d180c1b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:47:58.628184 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e651bf8-5fc9-4783-83fc-8fc42d180c1b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:47:58.632320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e5757-9249-4f6f-a58c-ff7414d6703d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:47:58.632947 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e5757-9249-4f6f-a58c-ff7414d6703d admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:47:58.633228 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e5757-9249-4f6f-a58c-ff7414d6703d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:47:58.633502 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e5757-9249-4f6f-a58c-ff7414d6703d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:47:58.641107 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e5757-9249-4f6f-a58c-ff7414d6703d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:47:58.642910 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e5757-9249-4f6f-a58c-ff7414d6703d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:47:58.651293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 4/8] 10.222.0.23 () {58 vars in 1324 bytes} [Mon May 9 22:47:58 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 22:47:58.656702 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc074d4e-c19a-4ed0-97e2-898a724b560a None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:47:58.666285 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc074d4e-c19a-4ed0-97e2-898a724b560a None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:47:58.670701 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea570dfe-221b-48e2-b9d0-ff23ff7c963e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:47:58.671559 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea570dfe-221b-48e2-b9d0-ff23ff7c963e admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:47:58.671840 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea570dfe-221b-48e2-b9d0-ff23ff7c963e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:47:58.672117 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea570dfe-221b-48e2-b9d0-ff23ff7c963e admin admin] PATH_INFO: `/v3/users/873ba60d483a41a4b36c04c1aa8b34ad` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:47:58.673494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea570dfe-221b-48e2-b9d0-ff23ff7c963e admin admin] RBAC: Authorizing `identity:get_user(user_id=873ba60d483a41a4b36c04c1aa8b34ad)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:47:58.675346 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea570dfe-221b-48e2-b9d0-ff23ff7c963e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:47:58.676942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 5/9] 10.222.0.23 () {56 vars in 1198 bytes} [Mon May 9 22:47:58 2022] GET /identity/v3/users/873ba60d483a41a4b36c04c1aa8b34ad => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:00.968454 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59169e68-a65c-4681-837a-dce322effd92 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:00.968954 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59169e68-a65c-4681-837a-dce322effd92 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:00.968954 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59169e68-a65c-4681-837a-dce322effd92 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:00.970303 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 5/10] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:00.975396 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344cc5c5-c9d4-4f8a-aefe-15a29ed351ea None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:00.975831 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344cc5c5-c9d4-4f8a-aefe-15a29ed351ea None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:00.975980 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344cc5c5-c9d4-4f8a-aefe-15a29ed351ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:00.979443 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16be0b7-3b05-40f2-9ae8-de5c8098c467 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:00.979735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16be0b7-3b05-40f2-9ae8-de5c8098c467 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:00.980020 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16be0b7-3b05-40f2-9ae8-de5c8098c467 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:00.981286 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 6/11] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:48:00.986687 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683617a3-0b52-407e-84bd-2a2455680656 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:00.986978 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683617a3-0b52-407e-84bd-2a2455680656 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:00.987260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683617a3-0b52-407e-84bd-2a2455680656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:01.029344 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG passlib.handlers.bcrypt [None req-683617a3-0b52-407e-84bd-2a2455680656 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=79888) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 09 22:48:01.029777 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG passlib.handlers.bcrypt [None req-683617a3-0b52-407e-84bd-2a2455680656 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=79888) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 09 22:48:01.080754 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-683617a3-0b52-407e-84bd-2a2455680656 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:01.082418 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-683617a3-0b52-407e-84bd-2a2455680656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:01.102724 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-344cc5c5-c9d4-4f8a-aefe-15a29ed351ea None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:01.103898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-344cc5c5-c9d4-4f8a-aefe-15a29ed351ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:01.183244 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 6/12] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:48:01.186368 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 7/13] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:48:01.204415 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344cc5c5-c9d4-4f8a-aefe-15a29ed351ea None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:01.205720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683617a3-0b52-407e-84bd-2a2455680656 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:01.300381 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344cc5c5-c9d4-4f8a-aefe-15a29ed351ea None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:01.300592 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683617a3-0b52-407e-84bd-2a2455680656 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:01.331622 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a365ab-78e5-4c40-aba6-1853e2c544b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:01.331812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed078c2-d97a-4920-ab15-ee1c8e88df15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:01.332442 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed078c2-d97a-4920-ab15-ee1c8e88df15 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:01.332656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a365ab-78e5-4c40-aba6-1853e2c544b4 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:01.332753 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed078c2-d97a-4920-ab15-ee1c8e88df15 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:01.332944 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a365ab-78e5-4c40-aba6-1853e2c544b4 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:01.333041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed078c2-d97a-4920-ab15-ee1c8e88df15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:01.333220 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a365ab-78e5-4c40-aba6-1853e2c544b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:01.345234 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a365ab-78e5-4c40-aba6-1853e2c544b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:01.356950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-34a365ab-78e5-4c40-aba6-1853e2c544b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:48:01.356950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:01.356950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:01.356950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:01.356950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:01.356950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:01.356950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:01.356950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:01.356950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:01.356950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:01.356950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:01.356950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:01.356950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:01.356950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:01.356950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:01.356950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:01.357970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:01.357970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:01.357970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:01.357970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:01.357970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:01.357970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:01.357970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:01.357970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:01.357970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:01.357970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:01.357970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:01.357970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:48:01.357970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:01.357970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:01.357970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:01.357970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:01.359033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:01.359033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:01.359033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:01.359033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:01.359033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:01.359033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:01.359033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:01.359033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:01.359033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:01.359033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:01.359033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:48:01.359033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:48:01.359033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 8/14] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:48:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:48:01.364132 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a365ab-78e5-4c40-aba6-1853e2c544b4 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:01.397587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed078c2-d97a-4920-ab15-ee1c8e88df15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:01.399609 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed078c2-d97a-4920-ab15-ee1c8e88df15 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:01.457254 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a365ab-78e5-4c40-aba6-1853e2c544b4 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:01.489743 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba45723-3fe5-43e6-8be9-9bd3f0865ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:01.490416 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba45723-3fe5-43e6-8be9-9bd3f0865ae2 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:01.490703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba45723-3fe5-43e6-8be9-9bd3f0865ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:01.490929 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 7/15] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:48:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 22:48:01.491045 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba45723-3fe5-43e6-8be9-9bd3f0865ae2 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:01.491729 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba45723-3fe5-43e6-8be9-9bd3f0865ae2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:01.492043 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba45723-3fe5-43e6-8be9-9bd3f0865ae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:01.493517 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba45723-3fe5-43e6-8be9-9bd3f0865ae2 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:01.496519 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed078c2-d97a-4920-ab15-ee1c8e88df15 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:01.506952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 9/16] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:01.512325 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba45723-3fe5-43e6-8be9-9bd3f0865ae2 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:01.590666 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed078c2-d97a-4920-ab15-ee1c8e88df15 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:01.625932 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4dbcd6-a877-4cdf-bccc-ec64d802de31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:01.626710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4dbcd6-a877-4cdf-bccc-ec64d802de31 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:01.626989 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4dbcd6-a877-4cdf-bccc-ec64d802de31 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:01.627258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4dbcd6-a877-4cdf-bccc-ec64d802de31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:01.638604 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4dbcd6-a877-4cdf-bccc-ec64d802de31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:01.640110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4dbcd6-a877-4cdf-bccc-ec64d802de31 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:01.661185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-da4dbcd6-a877-4cdf-bccc-ec64d802de31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:48:01.661185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:01.661185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:01.661185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:01.661185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:01.661185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:01.661185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:01.661185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:01.661185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:01.661185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:01.661185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:01.661185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:01.661185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:48:01.661185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:48:01.661185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:48:01.662280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:48:01.662280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:01.662280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:01.662280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:01.662280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:01.662280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:01.662280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:01.662280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:01.662280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:01.662280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:01.662280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:01.662280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:01.662280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:01.662280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:01.662280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:01.662280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:01.662280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:01.663139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:48:01.663139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:48:01.663139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:48:01.663139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:01.663139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 8/17] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:48:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:48:01.668797 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4dbcd6-a877-4cdf-bccc-ec64d802de31 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:01.716026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba45723-3fe5-43e6-8be9-9bd3f0865ae2 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:01.747137 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6664f45-6cdc-48aa-a7e7-78ddaefbc942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:01.747991 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6664f45-6cdc-48aa-a7e7-78ddaefbc942 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:01.748267 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6664f45-6cdc-48aa-a7e7-78ddaefbc942 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:01.748534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6664f45-6cdc-48aa-a7e7-78ddaefbc942 None admin] PATH_INFO: `/v3/users/873ba60d483a41a4b36c04c1aa8b34ad` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:01.750055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6664f45-6cdc-48aa-a7e7-78ddaefbc942 None admin] RBAC: Authorizing `identity:get_user(user_id=873ba60d483a41a4b36c04c1aa8b34ad)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:01.751523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6664f45-6cdc-48aa-a7e7-78ddaefbc942 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:01.753124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 10/18] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:01 2022] GET /identity/v3/users/873ba60d483a41a4b36c04c1aa8b34ad => generated 247 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:01.758953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6664f45-6cdc-48aa-a7e7-78ddaefbc942 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:01.765190 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4dbcd6-a877-4cdf-bccc-ec64d802de31 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:01.797014 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff3ab2-90f9-48e2-bcb6-94fc93168779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:01.797665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff3ab2-90f9-48e2-bcb6-94fc93168779 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:01.797948 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff3ab2-90f9-48e2-bcb6-94fc93168779 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:01.798258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff3ab2-90f9-48e2-bcb6-94fc93168779 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:01.798977 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff3ab2-90f9-48e2-bcb6-94fc93168779 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:01.799282 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff3ab2-90f9-48e2-bcb6-94fc93168779 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:01.800658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff3ab2-90f9-48e2-bcb6-94fc93168779 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:01.817208 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 9/19] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:48:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:01.854141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6664f45-6cdc-48aa-a7e7-78ddaefbc942 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:01.888563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332f343-c866-4aaf-a59d-6bc149d806fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:01.889359 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332f343-c866-4aaf-a59d-6bc149d806fb None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:01.889640 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332f343-c866-4aaf-a59d-6bc149d806fb None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:01.889911 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332f343-c866-4aaf-a59d-6bc149d806fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:01.891136 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332f343-c866-4aaf-a59d-6bc149d806fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:01.892624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332f343-c866-4aaf-a59d-6bc149d806fb None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:01.894317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 11/20] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:48:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:01.901019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff3ab2-90f9-48e2-bcb6-94fc93168779 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:02.011210 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff3ab2-90f9-48e2-bcb6-94fc93168779 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:02.044066 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492cdc07-cdd9-4494-8afe-61dfa5187ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:02.044719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cdc07-cdd9-4494-8afe-61dfa5187ffe None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:02.044999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cdc07-cdd9-4494-8afe-61dfa5187ffe None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:02.045285 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cdc07-cdd9-4494-8afe-61dfa5187ffe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:02.046125 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492cdc07-cdd9-4494-8afe-61dfa5187ffe None admin] RBAC: Adding query filter params (role.id=bda550492f4f4c5c83f52fa89fc18a7d, scope.domain.id=default, user.id=873ba60d483a41a4b36c04c1aa8b34ad) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:02.046448 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492cdc07-cdd9-4494-8afe-61dfa5187ffe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:02.047898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492cdc07-cdd9-4494-8afe-61dfa5187ffe None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:02.054745 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 10/21] 10.222.0.23 () {56 vars in 1322 bytes} [Mon May 9 22:48:01 2022] GET /identity/v3/role_assignments?user.id=873ba60d483a41a4b36c04c1aa8b34ad&scope.domain.id=default&role.id=bda550492f4f4c5c83f52fa89fc18a7d => generated 233 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:04.458256 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519ff11-7829-4bf1-87fc-d5dc5069bf57 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:04.458256 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519ff11-7829-4bf1-87fc-d5dc5069bf57 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:04.458256 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519ff11-7829-4bf1-87fc-d5dc5069bf57 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:04.459209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 12/22] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:04.464573 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d63a4-8619-48fc-b790-1e2756af54d2 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:04.465323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d63a4-8619-48fc-b790-1e2756af54d2 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:04.465757 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d63a4-8619-48fc-b790-1e2756af54d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:04.487129 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-c03d63a4-8619-48fc-b790-1e2756af54d2 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:04.488554 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c03d63a4-8619-48fc-b790-1e2756af54d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:04.547114 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af63bcf-b5a7-4c2f-9827-31c44d974355 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:04.547662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af63bcf-b5a7-4c2f-9827-31c44d974355 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:04.548087 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af63bcf-b5a7-4c2f-9827-31c44d974355 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:04.549849 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 13/23] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:04.554697 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec613bc-dba1-409b-a3eb-7313257b6217 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:04.555682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec613bc-dba1-409b-a3eb-7313257b6217 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:04.556131 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec613bc-dba1-409b-a3eb-7313257b6217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:04.578323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 11/24] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:04.591211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-1ec613bc-dba1-409b-a3eb-7313257b6217 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:04.592437 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1ec613bc-dba1-409b-a3eb-7313257b6217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:04.597087 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d63a4-8619-48fc-b790-1e2756af54d2 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:04.701977 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d63a4-8619-48fc-b790-1e2756af54d2 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:04.729505 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 14/25] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:04.739451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eac989-a98e-4f5b-bd8c-97dad2aed972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:04.740501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eac989-a98e-4f5b-bd8c-97dad2aed972 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:04.740958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eac989-a98e-4f5b-bd8c-97dad2aed972 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:04.741393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eac989-a98e-4f5b-bd8c-97dad2aed972 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:04.754362 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eac989-a98e-4f5b-bd8c-97dad2aed972 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:04.755399 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec613bc-dba1-409b-a3eb-7313257b6217 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:04.763824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-d8eac989-a98e-4f5b-bd8c-97dad2aed972 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:48:04.763824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:04.763824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:04.763824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:04.763824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:04.763824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:04.763824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:04.763824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:04.763824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:04.763824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:04.763824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:04.763824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:04.763824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:04.763824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:04.763824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:04.763824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:04.764886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:04.764886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:04.764886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:04.764886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:04.764886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:04.764886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:04.764886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:04.764886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:04.764886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:04.764886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:04.764886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:04.764886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:04.764886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:04.764886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:04.764886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:04.764886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:04.765819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:04.765819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:04.765819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:04.765819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:04.765819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:04.765819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:04.765819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:04.765819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:04.765819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:04.765819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:04.765819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:48:04.765819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:04.772458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 12/26] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:48:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:04.774642 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eac989-a98e-4f5b-bd8c-97dad2aed972 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:04.882282 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eac989-a98e-4f5b-bd8c-97dad2aed972 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:04.918452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40e38d-4f72-4539-9f6f-64122ca46fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:04.919355 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40e38d-4f72-4539-9f6f-64122ca46fcb None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:04.919783 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40e38d-4f72-4539-9f6f-64122ca46fcb None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:04.920204 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40e38d-4f72-4539-9f6f-64122ca46fcb None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:04.921089 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40e38d-4f72-4539-9f6f-64122ca46fcb None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:04.921629 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40e38d-4f72-4539-9f6f-64122ca46fcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:04.923242 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40e38d-4f72-4539-9f6f-64122ca46fcb None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:04.937801 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec613bc-dba1-409b-a3eb-7313257b6217 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:04.940368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 13/27] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:04.942516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40e38d-4f72-4539-9f6f-64122ca46fcb None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:04.993025 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0ec09-e482-4dbe-a163-9c3565b7ea77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:04.994044 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0ec09-e482-4dbe-a163-9c3565b7ea77 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:04.994495 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0ec09-e482-4dbe-a163-9c3565b7ea77 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:04.994915 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0ec09-e482-4dbe-a163-9c3565b7ea77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:05.022009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0ec09-e482-4dbe-a163-9c3565b7ea77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:05.035931 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0ec09-e482-4dbe-a163-9c3565b7ea77 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:05.049960 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40e38d-4f72-4539-9f6f-64122ca46fcb None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:05.056129 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-c7a0ec09-e482-4dbe-a163-9c3565b7ea77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:48:05.056129 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:05.056129 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:05.056129 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:05.056129 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:05.056129 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:05.056129 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:05.056129 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:05.056129 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:05.056129 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:05.056129 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:05.056129 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:05.056129 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:48:05.056129 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:48:05.056129 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:48:05.056983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:48:05.056983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:05.056983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:05.056983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:05.056983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:05.056983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:05.056983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:05.056983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:05.056983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:48:05.056983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:05.056983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:05.056983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:05.056983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:05.056983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:05.056983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:05.056983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:05.056983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:05.057871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:48:05.057871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:48:05.057871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:48:05.057871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:48:05.060176 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 15/28] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:48:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:48:05.065495 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0ec09-e482-4dbe-a163-9c3565b7ea77 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:05.088665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06e8ee-004e-49a2-a79c-f6a40ac3c4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:05.089840 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06e8ee-004e-49a2-a79c-f6a40ac3c4c8 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:05.090377 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06e8ee-004e-49a2-a79c-f6a40ac3c4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:05.090850 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06e8ee-004e-49a2-a79c-f6a40ac3c4c8 None admin] PATH_INFO: `/v3/users/873ba60d483a41a4b36c04c1aa8b34ad` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:05.092262 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb06e8ee-004e-49a2-a79c-f6a40ac3c4c8 None admin] RBAC: Authorizing `identity:get_user(user_id=873ba60d483a41a4b36c04c1aa8b34ad)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:05.093913 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb06e8ee-004e-49a2-a79c-f6a40ac3c4c8 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:05.100122 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 14/29] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:04 2022] GET /identity/v3/users/873ba60d483a41a4b36c04c1aa8b34ad => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:05.101662 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102626f-7ac6-449d-a045-ffaf11582f13 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:05.102868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102626f-7ac6-449d-a045-ffaf11582f13 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:05.103608 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102626f-7ac6-449d-a045-ffaf11582f13 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:05.106571 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 15/30] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:05.107693 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102626f-7ac6-449d-a045-ffaf11582f13 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:05.213605 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102626f-7ac6-449d-a045-ffaf11582f13 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:05.250857 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c758f0-334e-45ef-bf16-a500e36961a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:05.252004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c758f0-334e-45ef-bf16-a500e36961a3 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:05.252485 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c758f0-334e-45ef-bf16-a500e36961a3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:05.252955 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c758f0-334e-45ef-bf16-a500e36961a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:05.254363 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c758f0-334e-45ef-bf16-a500e36961a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:05.256023 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c758f0-334e-45ef-bf16-a500e36961a3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:05.262212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 16/31] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:48:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:05.263710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173cf33e-9dd7-4812-b8bf-4f393e132eef None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:05.264197 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173cf33e-9dd7-4812-b8bf-4f393e132eef None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:05.264669 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173cf33e-9dd7-4812-b8bf-4f393e132eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:05.282342 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0ec09-e482-4dbe-a163-9c3565b7ea77 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:05.282966 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-173cf33e-9dd7-4812-b8bf-4f393e132eef None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:05.284352 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-173cf33e-9dd7-4812-b8bf-4f393e132eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:05.352039 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 17/32] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:05.353239 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173cf33e-9dd7-4812-b8bf-4f393e132eef None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:05.382825 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ca22b-49ca-4b45-be26-b2a059737434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:05.384078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ca22b-49ca-4b45-be26-b2a059737434 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:05.384367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ca22b-49ca-4b45-be26-b2a059737434 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:05.384649 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ca22b-49ca-4b45-be26-b2a059737434 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:05.385376 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ca22b-49ca-4b45-be26-b2a059737434 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:05.385835 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ca22b-49ca-4b45-be26-b2a059737434 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:05.387396 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ca22b-49ca-4b45-be26-b2a059737434 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:05.421561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 16/33] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:05.422659 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ca22b-49ca-4b45-be26-b2a059737434 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:05.456348 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173cf33e-9dd7-4812-b8bf-4f393e132eef None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:05.491052 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6f5af-dfdb-435d-bb06-bb0002362cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:05.492044 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6f5af-dfdb-435d-bb06-bb0002362cf3 None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:05.492564 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6f5af-dfdb-435d-bb06-bb0002362cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:05.492923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6f5af-dfdb-435d-bb06-bb0002362cf3 None admin] PATH_INFO: `/v3/domains/default/users/873ba60d483a41a4b36c04c1aa8b34ad/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:05.495064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6f5af-dfdb-435d-bb06-bb0002362cf3 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=873ba60d483a41a4b36c04c1aa8b34ad, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:05.496601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6f5af-dfdb-435d-bb06-bb0002362cf3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:05.549735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ca22b-49ca-4b45-be26-b2a059737434 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:05.584425 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ae5e7-1bdd-43ca-a197-ff2d65487609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:05.585406 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ae5e7-1bdd-43ca-a197-ff2d65487609 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:05.585891 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ae5e7-1bdd-43ca-a197-ff2d65487609 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:05.586936 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ae5e7-1bdd-43ca-a197-ff2d65487609 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:05.587851 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ae5e7-1bdd-43ca-a197-ff2d65487609 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:05.589485 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ae5e7-1bdd-43ca-a197-ff2d65487609 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:05.630258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 18/34] 10.222.0.23 () {58 vars in 1361 bytes} [Mon May 9 22:48:05 2022] PUT /identity/v3/domains/default/users/873ba60d483a41a4b36c04c1aa8b34ad/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:48:05.631852 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6f5af-dfdb-435d-bb06-bb0002362cf3 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:05.732867 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 17/35] 10.222.0.23 () {60 vars in 1131 bytes} [Mon May 9 22:48:05 2022] POST /identity/v3/roles => generated 225 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:48:05.735135 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c5187-3555-49b9-9fe8-67e3fed3f488 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:05.735639 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c5187-3555-49b9-9fe8-67e3fed3f488 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:05.736741 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c5187-3555-49b9-9fe8-67e3fed3f488 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:05.738498 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 18/36] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:05.748509 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23dbef-6c4e-40e9-8d6f-dea35d027372 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:05.749429 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23dbef-6c4e-40e9-8d6f-dea35d027372 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:05.749919 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23dbef-6c4e-40e9-8d6f-dea35d027372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:05.778595 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-7e23dbef-6c4e-40e9-8d6f-dea35d027372 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:05.781077 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7e23dbef-6c4e-40e9-8d6f-dea35d027372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:05.873424 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6f5af-dfdb-435d-bb06-bb0002362cf3 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:05.926054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 19/37] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:05.936464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31003e4-114f-4ed0-8b57-3321505cd332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:05.937432 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31003e4-114f-4ed0-8b57-3321505cd332 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:05.937866 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31003e4-114f-4ed0-8b57-3321505cd332 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:05.938332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31003e4-114f-4ed0-8b57-3321505cd332 None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:05.939207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31003e4-114f-4ed0-8b57-3321505cd332 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:05.941064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31003e4-114f-4ed0-8b57-3321505cd332 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:05.955983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e23dbef-6c4e-40e9-8d6f-dea35d027372 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:06.009662 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 19/38] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 9 22:48:05 2022] POST /identity/v3/projects => generated 301 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:48:06.035035 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b0900-94f9-41e8-9b6d-2c8c20ed2ebc None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:06.035555 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b0900-94f9-41e8-9b6d-2c8c20ed2ebc None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:06.035980 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b0900-94f9-41e8-9b6d-2c8c20ed2ebc None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:06.038068 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 20/39] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:48:06.045542 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28975fa9-0e47-4d2b-8148-10a979acac03 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:06.046224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28975fa9-0e47-4d2b-8148-10a979acac03 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:06.046710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28975fa9-0e47-4d2b-8148-10a979acac03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:06.071470 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-28975fa9-0e47-4d2b-8148-10a979acac03 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:06.073580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-28975fa9-0e47-4d2b-8148-10a979acac03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:06.099524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e23dbef-6c4e-40e9-8d6f-dea35d027372 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:06.145400 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d358cb8a-ea38-4096-97cd-83aa2e75af64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:06.146364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358cb8a-ea38-4096-97cd-83aa2e75af64 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:06.146794 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358cb8a-ea38-4096-97cd-83aa2e75af64 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:06.147247 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358cb8a-ea38-4096-97cd-83aa2e75af64 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:06.148186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d358cb8a-ea38-4096-97cd-83aa2e75af64 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:06.149980 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d358cb8a-ea38-4096-97cd-83aa2e75af64 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:06.186853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 20/40] 10.222.0.23 () {60 vars in 1131 bytes} [Mon May 9 22:48:05 2022] POST /identity/v3/roles => generated 219 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 22:48:06.207473 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 21/41] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:06.253614 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d358cb8a-ea38-4096-97cd-83aa2e75af64 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:06.365965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d358cb8a-ea38-4096-97cd-83aa2e75af64 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:06.381130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91171d2b-874d-4ebe-b8d1-828d8c596df0 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:06.381460 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91171d2b-874d-4ebe-b8d1-828d8c596df0 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:06.381741 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91171d2b-874d-4ebe-b8d1-828d8c596df0 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:06.383437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 22/42] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:06.388732 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacad9e-8334-46cd-addc-c04a3b193adb None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:06.389022 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacad9e-8334-46cd-addc-c04a3b193adb None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:06.389303 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacad9e-8334-46cd-addc-c04a3b193adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:06.404587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262fd53f-6f3d-400d-bcc7-d40fe734c9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:06.405318 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262fd53f-6f3d-400d-bcc7-d40fe734c9af None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:06.405600 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262fd53f-6f3d-400d-bcc7-d40fe734c9af None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:06.405881 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262fd53f-6f3d-400d-bcc7-d40fe734c9af None admin] PATH_INFO: `/v3/domains/default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:06.407154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262fd53f-6f3d-400d-bcc7-d40fe734c9af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:06.408636 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262fd53f-6f3d-400d-bcc7-d40fe734c9af None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:06.410574 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 21/43] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:48:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:06.416636 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262fd53f-6f3d-400d-bcc7-d40fe734c9af None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:06.419056 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-fcacad9e-8334-46cd-addc-c04a3b193adb None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:06.420544 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-fcacad9e-8334-46cd-addc-c04a3b193adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:06.507320 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 23/44] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:48:06.529951 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacad9e-8334-46cd-addc-c04a3b193adb None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:06.540434 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262fd53f-6f3d-400d-bcc7-d40fe734c9af None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:06.576486 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e85ce-56ac-4207-856a-76a21c62b714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:06.577260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e85ce-56ac-4207-856a-76a21c62b714 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:06.577421 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e85ce-56ac-4207-856a-76a21c62b714 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:06.577836 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e85ce-56ac-4207-856a-76a21c62b714 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:06.578885 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e85ce-56ac-4207-856a-76a21c62b714 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:06.580356 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e85ce-56ac-4207-856a-76a21c62b714 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:06.614234 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 22/45] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 9 22:48:06 2022] POST /identity/v3/projects => generated 312 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 22:48:06.639039 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacad9e-8334-46cd-addc-c04a3b193adb None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:06.674765 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26ee5e-4185-4440-a82b-96dda9ea923d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:06.675375 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26ee5e-4185-4440-a82b-96dda9ea923d None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:06.675659 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26ee5e-4185-4440-a82b-96dda9ea923d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:06.675935 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26ee5e-4185-4440-a82b-96dda9ea923d None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:06.676702 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b26ee5e-4185-4440-a82b-96dda9ea923d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:06.678207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b26ee5e-4185-4440-a82b-96dda9ea923d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:06.694704 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 24/46] 10.222.0.23 () {60 vars in 1131 bytes} [Mon May 9 22:48:06 2022] POST /identity/v3/roles => generated 223 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 22:48:08.230943 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87ffda-da1f-437e-87d6-4584602ebe76 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:08.231458 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87ffda-da1f-437e-87d6-4584602ebe76 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:08.231564 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87ffda-da1f-437e-87d6-4584602ebe76 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:08.232840 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 23/47] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:08.238407 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8ad81-d0a2-4025-bd9c-1b360cba0a6b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:08.238802 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8ad81-d0a2-4025-bd9c-1b360cba0a6b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:08.238980 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8ad81-d0a2-4025-bd9c-1b360cba0a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:08.255109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-adb8ad81-d0a2-4025-bd9c-1b360cba0a6b None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:08.256301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-adb8ad81-d0a2-4025-bd9c-1b360cba0a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:08.320213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 25/48] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:08.343430 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87ffda-da1f-437e-87d6-4584602ebe76 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:08.437676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87ffda-da1f-437e-87d6-4584602ebe76 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:08.469728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b4322-684a-418f-8139-9efc4045dd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:08.470546 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b4322-684a-418f-8139-9efc4045dd96 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:08.470826 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b4322-684a-418f-8139-9efc4045dd96 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:08.471100 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b4322-684a-418f-8139-9efc4045dd96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:08.480666 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b4322-684a-418f-8139-9efc4045dd96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:08.488449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-a30b4322-684a-418f-8139-9efc4045dd96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:48:08.488449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:08.488449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:08.488449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:08.488449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:08.488449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:08.488449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:08.488449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:08.488449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:08.488449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:08.488449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:08.488449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:08.488449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:08.488449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:08.488449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:08.488449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:08.489395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:08.489395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:08.489395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:08.489395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:08.489395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:08.489395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:08.489395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:08.489395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:08.489395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:08.489395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:08.489395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:08.489395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:48:08.489395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:08.489395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:08.489395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:08.489395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:08.490201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:08.490201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:08.490201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:08.490201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:08.490201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:08.490201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:08.490201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:08.490201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:08.490201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:08.490201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:08.490201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:48:08.490201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:48:08.490201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 24/49] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:48:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:08.495257 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8ad81-d0a2-4025-bd9c-1b360cba0a6b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:08.585898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8ad81-d0a2-4025-bd9c-1b360cba0a6b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:08.617475 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01039f6-f20a-43fe-a982-cd0b6c75da16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:08.618139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01039f6-f20a-43fe-a982-cd0b6c75da16 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:08.618428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01039f6-f20a-43fe-a982-cd0b6c75da16 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:08.618698 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01039f6-f20a-43fe-a982-cd0b6c75da16 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:08.619440 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01039f6-f20a-43fe-a982-cd0b6c75da16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:08.619741 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01039f6-f20a-43fe-a982-cd0b6c75da16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:08.621132 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01039f6-f20a-43fe-a982-cd0b6c75da16 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:08.629147 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 26/50] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:08.635137 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b4322-684a-418f-8139-9efc4045dd96 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:08.735198 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b4322-684a-418f-8139-9efc4045dd96 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:08.767782 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc4ee8-5d26-4b32-a6f7-043c86e9fd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:08.768574 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc4ee8-5d26-4b32-a6f7-043c86e9fd02 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:08.768855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc4ee8-5d26-4b32-a6f7-043c86e9fd02 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:08.769127 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc4ee8-5d26-4b32-a6f7-043c86e9fd02 None admin] PATH_INFO: `/v3/users/873ba60d483a41a4b36c04c1aa8b34ad` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:08.770318 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc4ee8-5d26-4b32-a6f7-043c86e9fd02 None admin] RBAC: Authorizing `identity:get_user(user_id=873ba60d483a41a4b36c04c1aa8b34ad)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:08.771764 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc4ee8-5d26-4b32-a6f7-043c86e9fd02 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:08.773369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 25/51] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:08 2022] GET /identity/v3/users/873ba60d483a41a4b36c04c1aa8b34ad => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:08.778762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01039f6-f20a-43fe-a982-cd0b6c75da16 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:08.873662 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01039f6-f20a-43fe-a982-cd0b6c75da16 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:08.891308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db9de3-b225-4f28-aadc-46b7c8e133b9 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:08.891602 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db9de3-b225-4f28-aadc-46b7c8e133b9 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:08.891884 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db9de3-b225-4f28-aadc-46b7c8e133b9 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:08.893244 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 26/52] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:48:08.898105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2b7ac-c6d8-42d6-85a9-d68bc1ea693d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:08.898413 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2b7ac-c6d8-42d6-85a9-d68bc1ea693d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:08.898690 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2b7ac-c6d8-42d6-85a9-d68bc1ea693d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:08.905784 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a5a3c-a0b6-403d-89c9-caaa3b99d72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:08.906599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a5a3c-a0b6-403d-89c9-caaa3b99d72e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:08.906883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a5a3c-a0b6-403d-89c9-caaa3b99d72e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:08.907158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a5a3c-a0b6-403d-89c9-caaa3b99d72e None admin] PATH_INFO: `/v3/domains/default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:08.908312 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a5a3c-a0b6-403d-89c9-caaa3b99d72e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:08.909752 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a5a3c-a0b6-403d-89c9-caaa3b99d72e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:08.911380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 27/53] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:48:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:08.915102 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-49b2b7ac-c6d8-42d6-85a9-d68bc1ea693d None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:08.916277 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-49b2b7ac-c6d8-42d6-85a9-d68bc1ea693d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:08.917210 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a5a3c-a0b6-403d-89c9-caaa3b99d72e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:08.980731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 27/54] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:48:09.004102 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2b7ac-c6d8-42d6-85a9-d68bc1ea693d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:09.011804 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a5a3c-a0b6-403d-89c9-caaa3b99d72e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:09.043923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f495b-9508-469d-aadd-8256439544e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:09.044572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f495b-9508-469d-aadd-8256439544e3 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:09.044853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f495b-9508-469d-aadd-8256439544e3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:09.045128 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f495b-9508-469d-aadd-8256439544e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:09.045941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99f495b-9508-469d-aadd-8256439544e3 None admin] RBAC: Adding query filter params (role.id=bda550492f4f4c5c83f52fa89fc18a7d, scope.domain.id=default, user.id=873ba60d483a41a4b36c04c1aa8b34ad) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:09.046277 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99f495b-9508-469d-aadd-8256439544e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:09.047730 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99f495b-9508-469d-aadd-8256439544e3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:09.053199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 28/55] 10.222.0.23 () {56 vars in 1322 bytes} [Mon May 9 22:48:08 2022] GET /identity/v3/role_assignments?user.id=873ba60d483a41a4b36c04c1aa8b34ad&scope.domain.id=default&role.id=bda550492f4f4c5c83f52fa89fc18a7d => generated 530 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:09.101970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2b7ac-c6d8-42d6-85a9-d68bc1ea693d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:09.137044 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57004b-9fa1-4e26-a853-560aa69a86e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:09.137857 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57004b-9fa1-4e26-a853-560aa69a86e1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:09.138165 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57004b-9fa1-4e26-a853-560aa69a86e1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:09.138444 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57004b-9fa1-4e26-a853-560aa69a86e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:09.139674 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57004b-9fa1-4e26-a853-560aa69a86e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:09.141146 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57004b-9fa1-4e26-a853-560aa69a86e1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:09.142844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 28/56] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:48:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:09.149183 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f495b-9508-469d-aadd-8256439544e3 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:09.253298 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f495b-9508-469d-aadd-8256439544e3 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:09.286487 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee5683-fb00-4cf6-a65b-f448741aeb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:09.287154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee5683-fb00-4cf6-a65b-f448741aeb46 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:09.287437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee5683-fb00-4cf6-a65b-f448741aeb46 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:09.287713 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee5683-fb00-4cf6-a65b-f448741aeb46 None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:09.288420 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee5683-fb00-4cf6-a65b-f448741aeb46 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:09.289867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee5683-fb00-4cf6-a65b-f448741aeb46 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:09.314582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 29/57] 10.222.0.23 () {60 vars in 1140 bytes} [Mon May 9 22:48:09 2022] POST /identity/v3/projects => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:48:11.512939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612bc7f1-e034-4067-96e2-fe5c41d85adf None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:11.513460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612bc7f1-e034-4067-96e2-fe5c41d85adf None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:11.513548 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612bc7f1-e034-4067-96e2-fe5c41d85adf None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:11.514884 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 29/58] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:11.519938 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c450e8-edf3-4ee9-9696-a2f3a335faec None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:11.520342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c450e8-edf3-4ee9-9696-a2f3a335faec None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:11.520500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c450e8-edf3-4ee9-9696-a2f3a335faec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:11.537135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b8c450e8-edf3-4ee9-9696-a2f3a335faec None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:11.538344 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b8c450e8-edf3-4ee9-9696-a2f3a335faec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:11.600401 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 30/59] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:11.623892 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612bc7f1-e034-4067-96e2-fe5c41d85adf None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:11.720113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612bc7f1-e034-4067-96e2-fe5c41d85adf None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:11.751868 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157eb546-9dcf-4c65-beb4-1ad513719cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:11.752649 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157eb546-9dcf-4c65-beb4-1ad513719cd0 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:11.752934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157eb546-9dcf-4c65-beb4-1ad513719cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:11.753210 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157eb546-9dcf-4c65-beb4-1ad513719cd0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:11.754396 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157eb546-9dcf-4c65-beb4-1ad513719cd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:11.755834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157eb546-9dcf-4c65-beb4-1ad513719cd0 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:11.757422 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 30/60] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:48:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:11.762813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c450e8-edf3-4ee9-9696-a2f3a335faec None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:11.855392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c450e8-edf3-4ee9-9696-a2f3a335faec None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:11.886209 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611695b-3994-4b9f-844d-fdb0b6279f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:11.886860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611695b-3994-4b9f-844d-fdb0b6279f67 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:11.887143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611695b-3994-4b9f-844d-fdb0b6279f67 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:11.887419 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611695b-3994-4b9f-844d-fdb0b6279f67 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:11.888124 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1611695b-3994-4b9f-844d-fdb0b6279f67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:11.889566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1611695b-3994-4b9f-844d-fdb0b6279f67 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:11.913138 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 31/61] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 9 22:48:11 2022] POST /identity/v3/users => generated 275 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:48:15.167993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a276d-a6b5-4283-94ac-d88c11a43496 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:15.169013 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a276d-a6b5-4283-94ac-d88c11a43496 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:15.169510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a276d-a6b5-4283-94ac-d88c11a43496 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:15.171424 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 31/62] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:15.176475 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cc4e1-cf16-4f9a-b8dd-f4172045c116 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:15.177304 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cc4e1-cf16-4f9a-b8dd-f4172045c116 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:15.177818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cc4e1-cf16-4f9a-b8dd-f4172045c116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:15.195759 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-716cc4e1-cf16-4f9a-b8dd-f4172045c116 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:15.197313 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-716cc4e1-cf16-4f9a-b8dd-f4172045c116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:15.202653 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102f16c-8f2c-4da0-b480-eed4d7baf6b1 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:15.203188 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102f16c-8f2c-4da0-b480-eed4d7baf6b1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:15.203669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102f16c-8f2c-4da0-b480-eed4d7baf6b1 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:15.205222 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 32/63] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:48:15.210414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91a839-3787-4c16-9e93-bcf1a4b1794b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:15.213496 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91a839-3787-4c16-9e93-bcf1a4b1794b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:15.214024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91a839-3787-4c16-9e93-bcf1a4b1794b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:15.250108 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-bd91a839-3787-4c16-9e93-bcf1a4b1794b None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:15.251327 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-bd91a839-3787-4c16-9e93-bcf1a4b1794b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:15.293696 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 32/64] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:15.316743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cc4e1-cf16-4f9a-b8dd-f4172045c116 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:15.384234 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 33/65] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:15.418496 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91a839-3787-4c16-9e93-bcf1a4b1794b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:15.469814 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cc4e1-cf16-4f9a-b8dd-f4172045c116 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:15.520186 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d7956-cb7c-44fc-9cc0-7a2c30f2f5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:15.520991 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d7956-cb7c-44fc-9cc0-7a2c30f2f5d6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:15.521275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d7956-cb7c-44fc-9cc0-7a2c30f2f5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:15.521556 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d7956-cb7c-44fc-9cc0-7a2c30f2f5d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:15.544352 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d7956-cb7c-44fc-9cc0-7a2c30f2f5d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:15.553758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-114d7956-cb7c-44fc-9cc0-7a2c30f2f5d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:15.553758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:15.553758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:15.553758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:15.553758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:15.553758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:15.553758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:15.553758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:15.553758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:15.553758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:15.553758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:15.553758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:15.553758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:15.553758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:15.553758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:15.553758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:15.555467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:15.555467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:15.555467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:15.555467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:15.555467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:15.555467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:15.555467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:15.555467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:15.555467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:15.555467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:15.555467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:15.555467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:15.555467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:15.555467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:15.555467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:15.555467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:15.556349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:15.556349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:15.556349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:15.556349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:15.556349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:15.556349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:15.556349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:15.556349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:15.556349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:15.556349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:15.556349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:15.556349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:15.558780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 33/66] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:48:15.567219 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d7956-cb7c-44fc-9cc0-7a2c30f2f5d6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:15.598854 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91a839-3787-4c16-9e93-bcf1a4b1794b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:15.650854 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ea38e-1e93-44a7-85d5-4702e6c3a6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:15.651985 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ea38e-1e93-44a7-85d5-4702e6c3a6c0 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:15.652414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ea38e-1e93-44a7-85d5-4702e6c3a6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:15.652830 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ea38e-1e93-44a7-85d5-4702e6c3a6c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:15.675897 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ea38e-1e93-44a7-85d5-4702e6c3a6c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:15.691126 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d7956-cb7c-44fc-9cc0-7a2c30f2f5d6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:15.693253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-cf6ea38e-1e93-44a7-85d5-4702e6c3a6c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:48:15.693253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:15.693253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:15.693253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:15.693253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:15.693253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:15.693253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:15.693253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:15.693253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:15.693253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:15.693253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:15.693253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:15.693253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:15.693253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:15.693253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:15.693253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:15.694291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:15.694291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:15.694291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:15.694291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:15.694291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:15.694291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:15.694291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:15.694291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:15.694291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:15.694291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:15.694291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:15.694291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:48:15.694291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:15.694291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:15.694291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:15.694291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:15.695084 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:15.695084 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:15.695084 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:15.695084 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:15.695084 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:15.695084 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:15.695084 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:15.695084 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:15.695084 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:15.695084 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:15.695084 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:48:15.695084 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:48:15.698035 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 34/67] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:48:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:48:15.704482 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ea38e-1e93-44a7-85d5-4702e6c3a6c0 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:15.743307 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2305f8-fb13-4382-af3d-b2ca2ee81a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:15.744002 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2305f8-fb13-4382-af3d-b2ca2ee81a03 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:15.744289 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2305f8-fb13-4382-af3d-b2ca2ee81a03 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:15.744570 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2305f8-fb13-4382-af3d-b2ca2ee81a03 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:15.745336 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2305f8-fb13-4382-af3d-b2ca2ee81a03 None admin] RBAC: Adding query filter params (name=member) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:15.745644 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2305f8-fb13-4382-af3d-b2ca2ee81a03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:15.747080 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2305f8-fb13-4382-af3d-b2ca2ee81a03 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:15.765310 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 34/68] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:15.771068 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2305f8-fb13-4382-af3d-b2ca2ee81a03 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:15.840651 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ea38e-1e93-44a7-85d5-4702e6c3a6c0 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:15.875541 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611b384-caa1-48af-8678-c42d3e59b456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:15.876391 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611b384-caa1-48af-8678-c42d3e59b456 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:15.876770 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611b384-caa1-48af-8678-c42d3e59b456 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:15.877457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611b384-caa1-48af-8678-c42d3e59b456 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:15.877998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611b384-caa1-48af-8678-c42d3e59b456 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:15.878440 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611b384-caa1-48af-8678-c42d3e59b456 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:15.879930 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611b384-caa1-48af-8678-c42d3e59b456 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:15.889088 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 35/69] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:15.890501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2305f8-fb13-4382-af3d-b2ca2ee81a03 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:15.890690 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c460b-83c9-4ee1-8f3b-4a14c2d97a60 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:15.891078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c460b-83c9-4ee1-8f3b-4a14c2d97a60 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:15.891470 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c460b-83c9-4ee1-8f3b-4a14c2d97a60 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:15.897203 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 36/70] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:15.898138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c460b-83c9-4ee1-8f3b-4a14c2d97a60 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:15.925302 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9315c-27e4-4082-b74b-02bdac86c4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:15.926354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9315c-27e4-4082-b74b-02bdac86c4e3 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:15.926645 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9315c-27e4-4082-b74b-02bdac86c4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:15.926926 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9315c-27e4-4082-b74b-02bdac86c4e3 None admin] PATH_INFO: `/v3/users/11ec991028e64f3db9ef62861c931821` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:15.940634 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9315c-27e4-4082-b74b-02bdac86c4e3 None admin] RBAC: Authorizing `identity:get_user(user_id=11ec991028e64f3db9ef62861c931821)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:15.942159 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9315c-27e4-4082-b74b-02bdac86c4e3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:15.944015 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 35/71] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:15 2022] GET /identity/v3/users/11ec991028e64f3db9ef62861c931821 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:15.947531 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0611b6-8785-478a-97a5-f71a4a6f50ea None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:15.947827 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0611b6-8785-478a-97a5-f71a4a6f50ea None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:15.948112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0611b6-8785-478a-97a5-f71a4a6f50ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:15.965379 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-9b0611b6-8785-478a-97a5-f71a4a6f50ea None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:15.966593 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9b0611b6-8785-478a-97a5-f71a4a6f50ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:16.000672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c460b-83c9-4ee1-8f3b-4a14c2d97a60 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:16.029489 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 36/72] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:16.030863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ca050-fe0a-4e8a-9e35-fe33157d3c2a None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:16.031156 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ca050-fe0a-4e8a-9e35-fe33157d3c2a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:16.031435 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ca050-fe0a-4e8a-9e35-fe33157d3c2a None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:16.032690 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 37/73] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:16.033390 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ca050-fe0a-4e8a-9e35-fe33157d3c2a None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:16.040209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3dee3-04fb-4665-a9b4-a165725ae0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:16.041183 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3dee3-04fb-4665-a9b4-a165725ae0c0 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:16.041371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3dee3-04fb-4665-a9b4-a165725ae0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:16.041629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3dee3-04fb-4665-a9b4-a165725ae0c0 None admin] PATH_INFO: `/v3/users/873ba60d483a41a4b36c04c1aa8b34ad` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:16.042853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd3dee3-04fb-4665-a9b4-a165725ae0c0 None admin] RBAC: Authorizing `identity:get_user(user_id=873ba60d483a41a4b36c04c1aa8b34ad)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:16.044312 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd3dee3-04fb-4665-a9b4-a165725ae0c0 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:16.050499 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 37/74] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:15 2022] GET /identity/v3/users/873ba60d483a41a4b36c04c1aa8b34ad => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:16.051874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52fcc9-e441-46c7-8552-4bd683032674 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:16.052166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52fcc9-e441-46c7-8552-4bd683032674 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:16.052445 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52fcc9-e441-46c7-8552-4bd683032674 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:16.053683 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 38/75] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:16.055353 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc440b3-b6a3-461d-8650-91025e7f0ff5 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:16.055705 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc440b3-b6a3-461d-8650-91025e7f0ff5 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:16.055997 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc440b3-b6a3-461d-8650-91025e7f0ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:16.072591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-8dc440b3-b6a3-461d-8650-91025e7f0ff5 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:16.073973 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-8dc440b3-b6a3-461d-8650-91025e7f0ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:16.138007 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 39/76] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:16.150325 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc440b3-b6a3-461d-8650-91025e7f0ff5 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:16.151020 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ca050-fe0a-4e8a-9e35-fe33157d3c2a None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:16.187766 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b3598-1ead-4c98-a36b-8dbf2eca19ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:16.188363 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b3598-1ead-4c98-a36b-8dbf2eca19ef None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:16.188615 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b3598-1ead-4c98-a36b-8dbf2eca19ef None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:16.188896 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b3598-1ead-4c98-a36b-8dbf2eca19ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:16.231185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc440b3-b6a3-461d-8650-91025e7f0ff5 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:16.252048 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b3598-1ead-4c98-a36b-8dbf2eca19ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:16.253523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b3598-1ead-4c98-a36b-8dbf2eca19ef None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:16.262497 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea2396-da14-4616-bb39-174388fd6eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:16.263209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea2396-da14-4616-bb39-174388fd6eee None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:16.263563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea2396-da14-4616-bb39-174388fd6eee None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:16.263910 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea2396-da14-4616-bb39-174388fd6eee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:16.325949 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea2396-da14-4616-bb39-174388fd6eee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:16.327425 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea2396-da14-4616-bb39-174388fd6eee None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:16.345187 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 38/77] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:48:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:48:16.346008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b3598-1ead-4c98-a36b-8dbf2eca19ef None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:16.421023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 40/78] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:48:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:48:16.422336 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f5a0f-0f4a-4b6a-9c8c-23e06fe9cc3f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:16.422628 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f5a0f-0f4a-4b6a-9c8c-23e06fe9cc3f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:16.422909 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f5a0f-0f4a-4b6a-9c8c-23e06fe9cc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:16.438318 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-921f5a0f-0f4a-4b6a-9c8c-23e06fe9cc3f None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:16.438475 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b3598-1ead-4c98-a36b-8dbf2eca19ef None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:16.439427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-921f5a0f-0f4a-4b6a-9c8c-23e06fe9cc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:16.470517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ded507-f96e-4189-a477-7930c0cadadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:16.471316 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ded507-f96e-4189-a477-7930c0cadadc None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:16.471600 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ded507-f96e-4189-a477-7930c0cadadc None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:16.471878 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ded507-f96e-4189-a477-7930c0cadadc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:16.481943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ded507-f96e-4189-a477-7930c0cadadc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:16.490180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-a0ded507-f96e-4189-a477-7930c0cadadc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:48:16.490180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:16.490180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:16.490180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:16.490180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:16.490180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:16.490180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:16.490180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:16.490180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:16.490180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:16.490180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:16.490180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:16.490180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:16.490180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:16.490180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:16.490180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:16.491012 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:16.491012 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:16.491012 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:16.491012 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:16.491012 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:16.491012 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:16.491012 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:16.491012 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:16.491012 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:16.491012 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:16.491012 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:16.491012 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:16.491012 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:16.491012 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:16.491012 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:16.491012 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:16.491780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:16.491780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:16.491780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:16.491780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:16.491780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:16.491780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:16.491780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:16.491780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:16.491780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:16.491780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:16.491780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:48:16.491780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:16.491780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 39/79] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 9 22:48:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:16.492919 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ded507-f96e-4189-a477-7930c0cadadc None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:16.518142 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 41/80] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:16.518142 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f5a0f-0f4a-4b6a-9c8c-23e06fe9cc3f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:16.591245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ded507-f96e-4189-a477-7930c0cadadc None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:16.596286 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f5a0f-0f4a-4b6a-9c8c-23e06fe9cc3f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:16.623994 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643b7b8-b1d1-4a6f-b7f9-f80ed2438a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:16.624475 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643b7b8-b1d1-4a6f-b7f9-f80ed2438a9e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:16.624757 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643b7b8-b1d1-4a6f-b7f9-f80ed2438a9e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:16.625030 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643b7b8-b1d1-4a6f-b7f9-f80ed2438a9e None admin] PATH_INFO: `/v3/domains/default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:16.626241 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643b7b8-b1d1-4a6f-b7f9-f80ed2438a9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:16.627753 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643b7b8-b1d1-4a6f-b7f9-f80ed2438a9e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:16.629176 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3d5c4-4c46-4651-a9d5-3c72436a1e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:16.629297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 40/81] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:48:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:16.630030 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3d5c4-4c46-4651-a9d5-3c72436a1e81 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:16.630143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643b7b8-b1d1-4a6f-b7f9-f80ed2438a9e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:16.630408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3d5c4-4c46-4651-a9d5-3c72436a1e81 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:16.630684 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3d5c4-4c46-4651-a9d5-3c72436a1e81 None admin] PATH_INFO: `/v3/projects/1a6770d749f644eea4fd1584b1503d28` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:16.631874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3d5c4-4c46-4651-a9d5-3c72436a1e81 None admin] RBAC: Authorizing `identity:get_project(project_id=1a6770d749f644eea4fd1584b1503d28)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:16.633356 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3d5c4-4c46-4651-a9d5-3c72436a1e81 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:16.635249 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 42/82] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:16 2022] GET /identity/v3/projects/1a6770d749f644eea4fd1584b1503d28 => generated 298 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:16.636011 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3d5c4-4c46-4651-a9d5-3c72436a1e81 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:16.728629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3d5c4-4c46-4651-a9d5-3c72436a1e81 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:16.729634 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643b7b8-b1d1-4a6f-b7f9-f80ed2438a9e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:16.760918 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74238349-8511-455b-af6f-dc8b9d4a7545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:16.761234 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afa3cd-78f7-4bdc-a394-0b17be778a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:16.761584 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74238349-8511-455b-af6f-dc8b9d4a7545 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:16.761868 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74238349-8511-455b-af6f-dc8b9d4a7545 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:16.762081 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afa3cd-78f7-4bdc-a394-0b17be778a63 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:16.762208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74238349-8511-455b-af6f-dc8b9d4a7545 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:16.762392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afa3cd-78f7-4bdc-a394-0b17be778a63 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:16.762665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afa3cd-78f7-4bdc-a394-0b17be778a63 None admin] PATH_INFO: `/v3/projects/1a6770d749f644eea4fd1584b1503d28` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:16.762907 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74238349-8511-455b-af6f-dc8b9d4a7545 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:16.763221 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74238349-8511-455b-af6f-dc8b9d4a7545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:16.763846 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9afa3cd-78f7-4bdc-a394-0b17be778a63 None admin] RBAC: Authorizing `identity:get_project(project_id=1a6770d749f644eea4fd1584b1503d28)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:16.764594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74238349-8511-455b-af6f-dc8b9d4a7545 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:16.765285 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9afa3cd-78f7-4bdc-a394-0b17be778a63 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:16.767001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 41/83] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:16 2022] GET /identity/v3/projects/1a6770d749f644eea4fd1584b1503d28 => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:16.767753 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afa3cd-78f7-4bdc-a394-0b17be778a63 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:16.777171 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 43/84] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:48:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:16.777171 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74238349-8511-455b-af6f-dc8b9d4a7545 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:16.859995 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afa3cd-78f7-4bdc-a394-0b17be778a63 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:16.864605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74238349-8511-455b-af6f-dc8b9d4a7545 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:16.891141 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ca74d-6592-4f44-b08f-978d477758e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:16.891907 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ca74d-6592-4f44-b08f-978d477758e6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:16.892193 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ca74d-6592-4f44-b08f-978d477758e6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:16.892465 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ca74d-6592-4f44-b08f-978d477758e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:16.896069 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86196b-5773-4684-8e73-d1a659999515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:16.896729 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86196b-5773-4684-8e73-d1a659999515 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:16.897014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86196b-5773-4684-8e73-d1a659999515 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:16.897295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86196b-5773-4684-8e73-d1a659999515 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:16.897997 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa86196b-5773-4684-8e73-d1a659999515 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:16.900025 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa86196b-5773-4684-8e73-d1a659999515 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:16.903137 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ca74d-6592-4f44-b08f-978d477758e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:16.910736 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-a66ca74d-6592-4f44-b08f-978d477758e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:16.910736 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:16.910736 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:16.910736 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:16.910736 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:16.910736 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:16.910736 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:16.910736 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:16.910736 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:16.910736 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:16.910736 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:16.910736 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:16.910736 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:16.910736 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:16.910736 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:16.910736 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:16.911762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:16.911762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:16.911762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:16.911762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:16.911762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:16.911762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:16.911762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:16.911762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:16.911762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:16.911762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:16.911762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:16.911762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:16.911762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:16.911762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:16.911762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:16.911762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:16.913368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:16.913368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:16.913368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:16.913368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:16.913368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:16.913368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:16.913368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:16.913368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:16.913368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:16.913368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:16.913368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:16.913368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:16.913368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 42/85] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:16.913368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ca74d-6592-4f44-b08f-978d477758e6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:16.926239 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 44/86] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 9 22:48:16 2022] POST /identity/v3/users => generated 289 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:48:16.927055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86196b-5773-4684-8e73-d1a659999515 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:17.020368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ca74d-6592-4f44-b08f-978d477758e6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:17.031612 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86196b-5773-4684-8e73-d1a659999515 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:17.057057 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc69ea-ec09-435f-bee8-eedc48a15db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:17.057732 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc69ea-ec09-435f-bee8-eedc48a15db7 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:17.058019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc69ea-ec09-435f-bee8-eedc48a15db7 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:17.058326 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc69ea-ec09-435f-bee8-eedc48a15db7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:17.059184 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc69ea-ec09-435f-bee8-eedc48a15db7 None admin] RBAC: Adding query filter params (role.id=3631750e6bf34730aa3557cd90aa6108, scope.project.id=1a6770d749f644eea4fd1584b1503d28, user.id=11ec991028e64f3db9ef62861c931821) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:17.059495 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc69ea-ec09-435f-bee8-eedc48a15db7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:17.061041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc69ea-ec09-435f-bee8-eedc48a15db7 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:17.067656 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 43/87] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:48:16 2022] GET /identity/v3/role_assignments?user.id=11ec991028e64f3db9ef62861c931821&scope.project.id=1a6770d749f644eea4fd1584b1503d28&role.id=3631750e6bf34730aa3557cd90aa6108 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:17.068653 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc69ea-ec09-435f-bee8-eedc48a15db7 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:17.077565 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b2aa6-cc3f-4429-841b-0b2146350cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:17.078621 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b2aa6-cc3f-4429-841b-0b2146350cef None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:17.079066 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b2aa6-cc3f-4429-841b-0b2146350cef None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:17.079540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b2aa6-cc3f-4429-841b-0b2146350cef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:17.080585 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b2aa6-cc3f-4429-841b-0b2146350cef None admin] RBAC: Adding query filter params (role.id=bda550492f4f4c5c83f52fa89fc18a7d, scope.project.id=1a6770d749f644eea4fd1584b1503d28, user.id=873ba60d483a41a4b36c04c1aa8b34ad) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:17.081068 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b2aa6-cc3f-4429-841b-0b2146350cef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:17.082848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b2aa6-cc3f-4429-841b-0b2146350cef None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:17.095725 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 45/88] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:48:16 2022] GET /identity/v3/role_assignments?user.id=873ba60d483a41a4b36c04c1aa8b34ad&scope.project.id=1a6770d749f644eea4fd1584b1503d28&role.id=bda550492f4f4c5c83f52fa89fc18a7d => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:17.096866 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b2aa6-cc3f-4429-841b-0b2146350cef None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:17.207736 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc69ea-ec09-435f-bee8-eedc48a15db7 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:17.225979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b2aa6-cc3f-4429-841b-0b2146350cef None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:17.255684 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ac68f-e853-430b-80d0-f90052ccf95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:17.257421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ac68f-e853-430b-80d0-f90052ccf95e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:17.257893 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ac68f-e853-430b-80d0-f90052ccf95e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:17.258504 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ac68f-e853-430b-80d0-f90052ccf95e None admin] PATH_INFO: `/v3/users/11ec991028e64f3db9ef62861c931821` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:17.260606 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ac68f-e853-430b-80d0-f90052ccf95e None admin] RBAC: Authorizing `identity:get_user(user_id=11ec991028e64f3db9ef62861c931821)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:17.262354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ac68f-e853-430b-80d0-f90052ccf95e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:17.264766 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 44/89] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:17 2022] GET /identity/v3/users/11ec991028e64f3db9ef62861c931821 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:17.270957 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ac68f-e853-430b-80d0-f90052ccf95e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:17.272858 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ded09-39f4-43b6-b1ab-5e024694c255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:17.274233 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ded09-39f4-43b6-b1ab-5e024694c255 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:17.274593 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ded09-39f4-43b6-b1ab-5e024694c255 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:17.274946 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ded09-39f4-43b6-b1ab-5e024694c255 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:17.275822 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ded09-39f4-43b6-b1ab-5e024694c255 None admin] RBAC: Adding query filter params (name=member) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:17.276217 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ded09-39f4-43b6-b1ab-5e024694c255 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:17.277912 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ded09-39f4-43b6-b1ab-5e024694c255 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:17.288305 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 46/90] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:17.294688 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ded09-39f4-43b6-b1ab-5e024694c255 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:17.406994 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ac68f-e853-430b-80d0-f90052ccf95e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:17.442853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bbc1c0-562d-4ae7-ba4b-458861986fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:17.443507 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbc1c0-562d-4ae7-ba4b-458861986fdf None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:17.443785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbc1c0-562d-4ae7-ba4b-458861986fdf None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:17.444058 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbc1c0-562d-4ae7-ba4b-458861986fdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:17.451775 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ded09-39f4-43b6-b1ab-5e024694c255 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:17.496753 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29913f5-b840-4ce6-94c5-d4e0fd3e3182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:17.497635 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29913f5-b840-4ce6-94c5-d4e0fd3e3182 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:17.497922 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29913f5-b840-4ce6-94c5-d4e0fd3e3182 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:17.500386 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29913f5-b840-4ce6-94c5-d4e0fd3e3182 None admin] PATH_INFO: `/v3/users/11ec991028e64f3db9ef62861c931821` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:17.501724 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29913f5-b840-4ce6-94c5-d4e0fd3e3182 None admin] RBAC: Authorizing `identity:get_user(user_id=11ec991028e64f3db9ef62861c931821)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:17.503278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29913f5-b840-4ce6-94c5-d4e0fd3e3182 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:17.505051 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 47/91] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:17 2022] GET /identity/v3/users/11ec991028e64f3db9ef62861c931821 => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:17.511105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29913f5-b840-4ce6-94c5-d4e0fd3e3182 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:17.515995 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bbc1c0-562d-4ae7-ba4b-458861986fdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:17.517686 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bbc1c0-562d-4ae7-ba4b-458861986fdf None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:17.624214 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 45/92] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:48:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:48:17.630606 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bbc1c0-562d-4ae7-ba4b-458861986fdf None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:17.633425 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29913f5-b840-4ce6-94c5-d4e0fd3e3182 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:17.685288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bae163-982c-4dc4-add9-35ef75140c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:17.686024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bae163-982c-4dc4-add9-35ef75140c25 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:17.686345 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bae163-982c-4dc4-add9-35ef75140c25 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:17.686679 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bae163-982c-4dc4-add9-35ef75140c25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:17.794101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bbc1c0-562d-4ae7-ba4b-458861986fdf None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:17.796342 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bae163-982c-4dc4-add9-35ef75140c25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:17.797844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bae163-982c-4dc4-add9-35ef75140c25 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:17.862957 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa54c20-ebc3-43e8-9059-80636e099994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:17.863825 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa54c20-ebc3-43e8-9059-80636e099994 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:17.864110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa54c20-ebc3-43e8-9059-80636e099994 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:17.864388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa54c20-ebc3-43e8-9059-80636e099994 None admin] PATH_INFO: `/v3/projects/1a6770d749f644eea4fd1584b1503d28` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:17.868998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa54c20-ebc3-43e8-9059-80636e099994 None admin] RBAC: Authorizing `identity:get_project(project_id=1a6770d749f644eea4fd1584b1503d28)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:17.870497 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa54c20-ebc3-43e8-9059-80636e099994 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:17.874290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 46/93] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:17 2022] GET /identity/v3/projects/1a6770d749f644eea4fd1584b1503d28 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:17.879201 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa54c20-ebc3-43e8-9059-80636e099994 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:17.993732 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 48/94] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:48:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:48:17.999051 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bae163-982c-4dc4-add9-35ef75140c25 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:18.102415 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa54c20-ebc3-43e8-9059-80636e099994 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:18.183963 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030160e2-2e81-4180-ad5f-138ea8972ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:18.184551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030160e2-2e81-4180-ad5f-138ea8972ca4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:18.184835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030160e2-2e81-4180-ad5f-138ea8972ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:18.185110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030160e2-2e81-4180-ad5f-138ea8972ca4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:18.185948 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030160e2-2e81-4180-ad5f-138ea8972ca4 None admin] RBAC: Adding query filter params (role.id=46ddc3cf4b124399a275bc639635e10a, scope.project.id=1a6770d749f644eea4fd1584b1503d28, user.id=11ec991028e64f3db9ef62861c931821) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:18.186289 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030160e2-2e81-4180-ad5f-138ea8972ca4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:18.187736 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030160e2-2e81-4180-ad5f-138ea8972ca4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:18.190027 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bae163-982c-4dc4-add9-35ef75140c25 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:18.198646 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 47/95] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:48:17 2022] GET /identity/v3/role_assignments?user.id=11ec991028e64f3db9ef62861c931821&scope.project.id=1a6770d749f644eea4fd1584b1503d28&role.id=46ddc3cf4b124399a275bc639635e10a => generated 259 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:18.252395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff0df3-3071-4db6-925b-88b4a6ef58b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:18.253354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff0df3-3071-4db6-925b-88b4a6ef58b7 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:18.253644 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff0df3-3071-4db6-925b-88b4a6ef58b7 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:18.253927 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff0df3-3071-4db6-925b-88b4a6ef58b7 None admin] PATH_INFO: `/v3/projects/f8933570c9dc44c8bb1ecbd74374889b` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:18.257109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff0df3-3071-4db6-925b-88b4a6ef58b7 None admin] RBAC: Authorizing `identity:get_project(project_id=f8933570c9dc44c8bb1ecbd74374889b)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:18.258678 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff0df3-3071-4db6-925b-88b4a6ef58b7 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:18.260762 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 49/96] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:17 2022] GET /identity/v3/projects/f8933570c9dc44c8bb1ecbd74374889b => generated 312 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:18.266454 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030160e2-2e81-4180-ad5f-138ea8972ca4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:18.378511 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030160e2-2e81-4180-ad5f-138ea8972ca4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:18.417616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c0664-4986-4342-ac23-284bd4820ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:18.418307 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c0664-4986-4342-ac23-284bd4820ce6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:18.418591 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c0664-4986-4342-ac23-284bd4820ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:18.418868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c0664-4986-4342-ac23-284bd4820ce6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:18.419697 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c0664-4986-4342-ac23-284bd4820ce6 None admin] RBAC: Adding query filter params (role.id=3631750e6bf34730aa3557cd90aa6108, scope.project.id=f8933570c9dc44c8bb1ecbd74374889b, user.id=11ec991028e64f3db9ef62861c931821) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:18.420008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c0664-4986-4342-ac23-284bd4820ce6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:18.421446 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c0664-4986-4342-ac23-284bd4820ce6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:18.427397 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 48/97] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:48:18 2022] GET /identity/v3/role_assignments?user.id=11ec991028e64f3db9ef62861c931821&scope.project.id=f8933570c9dc44c8bb1ecbd74374889b&role.id=3631750e6bf34730aa3557cd90aa6108 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:20.210164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3b510-4a79-4371-adb4-35a29e4626a0 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:20.210717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3b510-4a79-4371-adb4-35a29e4626a0 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:20.210791 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3b510-4a79-4371-adb4-35a29e4626a0 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:20.212118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 50/98] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:20.216191 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24051f11-ecd6-4df9-acf1-ba1b16c56d00 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:20.217963 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c18a9f-6b8f-4f55-8571-5eae0ce1fad6 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:20.218503 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c18a9f-6b8f-4f55-8571-5eae0ce1fad6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:20.218997 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c18a9f-6b8f-4f55-8571-5eae0ce1fad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:20.219473 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24051f11-ecd6-4df9-acf1-ba1b16c56d00 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:20.221268 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24051f11-ecd6-4df9-acf1-ba1b16c56d00 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:20.222890 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 49/99] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:20.235982 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2465ff3-1f53-4bb6-89ba-2a6106447caa None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:20.236591 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2465ff3-1f53-4bb6-89ba-2a6106447caa None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:20.237063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2465ff3-1f53-4bb6-89ba-2a6106447caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:20.247347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-69c18a9f-6b8f-4f55-8571-5eae0ce1fad6 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:20.248949 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-69c18a9f-6b8f-4f55-8571-5eae0ce1fad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:20.287256 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-f2465ff3-1f53-4bb6-89ba-2a6106447caa None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:20.288835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f2465ff3-1f53-4bb6-89ba-2a6106447caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:20.322748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 51/100] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:20.354689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c18a9f-6b8f-4f55-8571-5eae0ce1fad6 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:20.420803 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 50/101] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 09 22:48:20.481561 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2465ff3-1f53-4bb6-89ba-2a6106447caa None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:20.506976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c18a9f-6b8f-4f55-8571-5eae0ce1fad6 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:20.542309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eea47e-6091-487b-89a4-6369b8007a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:20.543478 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eea47e-6091-487b-89a4-6369b8007a70 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:20.543932 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eea47e-6091-487b-89a4-6369b8007a70 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:20.544350 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eea47e-6091-487b-89a4-6369b8007a70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:20.555562 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eea47e-6091-487b-89a4-6369b8007a70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:20.563638 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-66eea47e-6091-487b-89a4-6369b8007a70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:48:20.563638 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:20.563638 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:20.563638 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:20.563638 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:20.563638 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:20.563638 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:20.563638 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:20.563638 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:20.563638 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:20.563638 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:20.563638 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:20.563638 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:20.563638 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:20.563638 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:20.563638 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:20.564563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:20.564563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:20.564563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:20.564563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:20.564563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:20.564563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:20.564563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:20.564563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:20.564563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:20.564563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:20.564563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:20.564563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:48:20.564563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:20.564563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:20.564563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:20.564563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:20.565346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:20.565346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:20.565346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:20.565346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:20.565346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:20.565346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:20.565346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:20.565346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:20.565346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:20.565346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:20.565346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:48:20.565346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:48:20.567807 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 52/102] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:48:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:48:20.575053 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eea47e-6091-487b-89a4-6369b8007a70 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:20.641337 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2465ff3-1f53-4bb6-89ba-2a6106447caa None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:20.683014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eea47e-6091-487b-89a4-6369b8007a70 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:20.692256 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6033c-dffa-4880-a2d7-060965326084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:20.693058 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6033c-dffa-4880-a2d7-060965326084 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:20.693343 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6033c-dffa-4880-a2d7-060965326084 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:20.693616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6033c-dffa-4880-a2d7-060965326084 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:20.717442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46567f99-bc4d-478b-a901-4060d13dd4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:20.718137 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46567f99-bc4d-478b-a901-4060d13dd4b4 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:20.718428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46567f99-bc4d-478b-a901-4060d13dd4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:20.718715 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46567f99-bc4d-478b-a901-4060d13dd4b4 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:20.719478 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46567f99-bc4d-478b-a901-4060d13dd4b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:20.719786 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46567f99-bc4d-478b-a901-4060d13dd4b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:20.721181 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46567f99-bc4d-478b-a901-4060d13dd4b4 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:20.723446 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6033c-dffa-4880-a2d7-060965326084 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:20.732514 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 53/103] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:20.737019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-88c6033c-dffa-4880-a2d7-060965326084 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:20.737019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:20.737019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:20.737019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:20.737019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:20.737019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:20.737019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:20.737019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:20.737019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:20.737019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:20.737019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:20.737019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:20.737019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:20.737019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:20.737019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:20.737019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:20.737905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:20.737905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:20.737905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:20.737905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:20.737905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:20.737905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:20.737905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:20.737905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:20.737905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:20.737905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:20.737905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:20.737905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:20.737905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:20.737905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:20.737905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:20.737905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:20.738729 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:20.738729 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:20.738729 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:20.738729 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:20.738729 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:20.738729 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:20.738729 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:20.738729 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:20.738729 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:20.738729 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:20.738729 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:20.738729 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:20.744002 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 51/104] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:48:20.745803 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46567f99-bc4d-478b-a901-4060d13dd4b4 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:20.749306 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6033c-dffa-4880-a2d7-060965326084 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:20.940887 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6033c-dffa-4880-a2d7-060965326084 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:20.968603 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46567f99-bc4d-478b-a901-4060d13dd4b4 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:20.999372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b9f1c-24de-43c0-9012-7c1c49f4161d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:20.999676 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b9f1c-24de-43c0-9012-7c1c49f4161d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:20.999847 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b9f1c-24de-43c0-9012-7c1c49f4161d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:21.006135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b9f1c-24de-43c0-9012-7c1c49f4161d None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:21.006135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b9f1c-24de-43c0-9012-7c1c49f4161d None admin] RBAC: Adding query filter params (name=member) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:21.006135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b9f1c-24de-43c0-9012-7c1c49f4161d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:21.006135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b9f1c-24de-43c0-9012-7c1c49f4161d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:21.015872 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 52/105] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:21.017241 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8abb7f2-62e6-4109-b7d7-f8ba036818d0 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:21.019066 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8abb7f2-62e6-4109-b7d7-f8ba036818d0 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:21.019066 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8abb7f2-62e6-4109-b7d7-f8ba036818d0 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:21.022476 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 53/106] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:21.025259 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8abb7f2-62e6-4109-b7d7-f8ba036818d0 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:21.049460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f18558-851f-428d-9f5a-5f77989409d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:21.050039 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f18558-851f-428d-9f5a-5f77989409d0 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:21.050234 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f18558-851f-428d-9f5a-5f77989409d0 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:21.051057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f18558-851f-428d-9f5a-5f77989409d0 None admin] PATH_INFO: `/v3/users/873ba60d483a41a4b36c04c1aa8b34ad` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:21.051703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f18558-851f-428d-9f5a-5f77989409d0 None admin] RBAC: Authorizing `identity:get_user(user_id=873ba60d483a41a4b36c04c1aa8b34ad)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:21.053159 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f18558-851f-428d-9f5a-5f77989409d0 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:21.055195 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 54/107] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:20 2022] GET /identity/v3/users/873ba60d483a41a4b36c04c1aa8b34ad => generated 247 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:21.056513 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb02f4b-5713-4c6b-a231-03b5008cafd6 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:21.058339 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb02f4b-5713-4c6b-a231-03b5008cafd6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:21.058339 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb02f4b-5713-4c6b-a231-03b5008cafd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:21.087070 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-ceb02f4b-5713-4c6b-a231-03b5008cafd6 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:21.088437 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ceb02f4b-5713-4c6b-a231-03b5008cafd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:21.188831 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 55/108] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:21.195790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb02f4b-5713-4c6b-a231-03b5008cafd6 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:21.261335 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8abb7f2-62e6-4109-b7d7-f8ba036818d0 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:21.330359 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb02f4b-5713-4c6b-a231-03b5008cafd6 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:21.334148 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37063d17-1396-4ed5-8755-616871d6fb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:21.334148 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37063d17-1396-4ed5-8755-616871d6fb75 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:21.334148 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37063d17-1396-4ed5-8755-616871d6fb75 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:21.334148 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37063d17-1396-4ed5-8755-616871d6fb75 None admin] PATH_INFO: `/v3/users/11ec991028e64f3db9ef62861c931821` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:21.335064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37063d17-1396-4ed5-8755-616871d6fb75 None admin] RBAC: Authorizing `identity:get_user(user_id=11ec991028e64f3db9ef62861c931821)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:21.336547 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37063d17-1396-4ed5-8755-616871d6fb75 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:21.338722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 54/109] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:21 2022] GET /identity/v3/users/11ec991028e64f3db9ef62861c931821 => generated 275 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:21.339966 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834bb783-6ebf-41c7-9e69-e0ebba3f8c0b None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:21.340258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834bb783-6ebf-41c7-9e69-e0ebba3f8c0b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:21.340921 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834bb783-6ebf-41c7-9e69-e0ebba3f8c0b None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:21.342554 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 55/110] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:21.343816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834bb783-6ebf-41c7-9e69-e0ebba3f8c0b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:21.376558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064dd00a-44d4-4df9-ad45-69fce1abf6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:21.377577 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064dd00a-44d4-4df9-ad45-69fce1abf6d2 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:21.378766 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064dd00a-44d4-4df9-ad45-69fce1abf6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:21.378766 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064dd00a-44d4-4df9-ad45-69fce1abf6d2 None admin] PATH_INFO: `/v3/projects/1a6770d749f644eea4fd1584b1503d28` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:21.382163 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064dd00a-44d4-4df9-ad45-69fce1abf6d2 None admin] RBAC: Authorizing `identity:get_project(project_id=1a6770d749f644eea4fd1584b1503d28)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:21.382346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064dd00a-44d4-4df9-ad45-69fce1abf6d2 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:21.385502 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 56/111] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:21 2022] GET /identity/v3/projects/1a6770d749f644eea4fd1584b1503d28 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:21.387493 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75798d91-7790-4480-b73c-a6141a4cdc64 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:21.387994 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75798d91-7790-4480-b73c-a6141a4cdc64 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:21.388495 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75798d91-7790-4480-b73c-a6141a4cdc64 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:21.394912 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 57/112] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:21.395808 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75798d91-7790-4480-b73c-a6141a4cdc64 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:21.470621 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834bb783-6ebf-41c7-9e69-e0ebba3f8c0b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:21.503754 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac084eea-1210-4851-8a9c-0f87fc12d292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:21.504548 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac084eea-1210-4851-8a9c-0f87fc12d292 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:21.504833 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac084eea-1210-4851-8a9c-0f87fc12d292 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:21.505109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac084eea-1210-4851-8a9c-0f87fc12d292 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:21.515003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac084eea-1210-4851-8a9c-0f87fc12d292 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:21.517020 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75798d91-7790-4480-b73c-a6141a4cdc64 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:21.522599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-ac084eea-1210-4851-8a9c-0f87fc12d292 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 22:48:21.522599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:21.522599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:21.522599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:21.522599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:21.522599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:21.522599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:21.522599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:21.522599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:21.522599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:21.522599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:21.522599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:21.522599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:21.522599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:21.522599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:21.522599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:21.524025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:21.524025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:21.524025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:21.524025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:21.524025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:21.524025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:21.524025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:21.524025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:21.524025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:21.524025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:21.524025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:21.524025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:21.524025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:21.524025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:21.524025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:21.524025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:21.524865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:21.524865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:21.524865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:21.524865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:21.524865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:21.524865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:21.524865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:21.524865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:21.524865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:21.524865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:21.524865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 22:48:21.524865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:21.524865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 56/113] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:21.526045 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500765f-15ec-432e-a8f5-363376f16708 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:21.526370 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500765f-15ec-432e-a8f5-363376f16708 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:21.526641 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500765f-15ec-432e-a8f5-363376f16708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:21.544203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d500765f-15ec-432e-a8f5-363376f16708 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:21.545486 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d500765f-15ec-432e-a8f5-363376f16708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:21.556974 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e011506b-bb9b-495e-8a81-d5c44fe21aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:21.557835 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011506b-bb9b-495e-8a81-d5c44fe21aa4 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:21.558133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011506b-bb9b-495e-8a81-d5c44fe21aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:21.558436 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011506b-bb9b-495e-8a81-d5c44fe21aa4 None admin] PATH_INFO: `/v3/projects/1a6770d749f644eea4fd1584b1503d28` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:21.559616 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e011506b-bb9b-495e-8a81-d5c44fe21aa4 None admin] RBAC: Authorizing `identity:get_project(project_id=1a6770d749f644eea4fd1584b1503d28)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:21.561111 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e011506b-bb9b-495e-8a81-d5c44fe21aa4 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:21.562866 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 58/114] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:21 2022] GET /identity/v3/projects/1a6770d749f644eea4fd1584b1503d28 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:21.563723 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e011506b-bb9b-495e-8a81-d5c44fe21aa4 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:21.625179 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 57/115] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:21.626289 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc803b-bcfb-447f-91c9-b7eac130d81c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:21.626580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc803b-bcfb-447f-91c9-b7eac130d81c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:21.626862 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc803b-bcfb-447f-91c9-b7eac130d81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:21.660170 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-8dbc803b-bcfb-447f-91c9-b7eac130d81c None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:21.661404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8dbc803b-bcfb-447f-91c9-b7eac130d81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:21.675100 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e011506b-bb9b-495e-8a81-d5c44fe21aa4 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:21.722463 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d9d8a-fbf1-4e02-b2b5-4949308b6647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:21.723365 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d9d8a-fbf1-4e02-b2b5-4949308b6647 None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:21.723646 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d9d8a-fbf1-4e02-b2b5-4949308b6647 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:21.723920 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d9d8a-fbf1-4e02-b2b5-4949308b6647 None admin] PATH_INFO: `/v3/projects/1a6770d749f644eea4fd1584b1503d28/users/873ba60d483a41a4b36c04c1aa8b34ad/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:21.725913 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d9d8a-fbf1-4e02-b2b5-4949308b6647 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a6770d749f644eea4fd1584b1503d28, user_id=873ba60d483a41a4b36c04c1aa8b34ad, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:21.728014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d9d8a-fbf1-4e02-b2b5-4949308b6647 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:21.738296 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 58/116] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:21.738296 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc803b-bcfb-447f-91c9-b7eac130d81c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:21.752238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 59/117] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:48:21 2022] PUT /identity/v3/projects/1a6770d749f644eea4fd1584b1503d28/users/873ba60d483a41a4b36c04c1aa8b34ad/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:48:21.753121 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d9d8a-fbf1-4e02-b2b5-4949308b6647 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:21.869835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc803b-bcfb-447f-91c9-b7eac130d81c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:21.882878 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d9d8a-fbf1-4e02-b2b5-4949308b6647 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:21.903954 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b332910-ba15-47dd-97f0-f222bcd21a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:21.904952 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b332910-ba15-47dd-97f0-f222bcd21a47 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:21.905241 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b332910-ba15-47dd-97f0-f222bcd21a47 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:21.905518 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b332910-ba15-47dd-97f0-f222bcd21a47 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:21.906316 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b332910-ba15-47dd-97f0-f222bcd21a47 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:21.906627 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b332910-ba15-47dd-97f0-f222bcd21a47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:21.908062 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b332910-ba15-47dd-97f0-f222bcd21a47 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:21.917976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 59/118] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:21.922333 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b332910-ba15-47dd-97f0-f222bcd21a47 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:21.926395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad580a-c2fe-4d41-ace1-d972c75e0da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:21.927307 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad580a-c2fe-4d41-ace1-d972c75e0da9 None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:21.927591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad580a-c2fe-4d41-ace1-d972c75e0da9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:21.927870 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad580a-c2fe-4d41-ace1-d972c75e0da9 None admin] PATH_INFO: `/v3/projects/1a6770d749f644eea4fd1584b1503d28/users/11ec991028e64f3db9ef62861c931821/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:21.930532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad580a-c2fe-4d41-ace1-d972c75e0da9 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a6770d749f644eea4fd1584b1503d28, user_id=11ec991028e64f3db9ef62861c931821, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:21.932020 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad580a-c2fe-4d41-ace1-d972c75e0da9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:21.951160 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 60/119] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:48:21 2022] PUT /identity/v3/projects/1a6770d749f644eea4fd1584b1503d28/users/11ec991028e64f3db9ef62861c931821/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:48:21.951992 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad580a-c2fe-4d41-ace1-d972c75e0da9 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:22.035613 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b332910-ba15-47dd-97f0-f222bcd21a47 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:22.074660 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc50e2-a15c-453a-8491-dc82a64688e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:22.075518 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc50e2-a15c-453a-8491-dc82a64688e6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:22.075803 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc50e2-a15c-453a-8491-dc82a64688e6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:22.076133 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc50e2-a15c-453a-8491-dc82a64688e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:22.077659 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdc50e2-a15c-453a-8491-dc82a64688e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:22.079385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdc50e2-a15c-453a-8491-dc82a64688e6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:22.081447 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 60/120] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:48:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:22.082940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353a772-609d-4008-b2a9-3bce22402fff None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:22.083282 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353a772-609d-4008-b2a9-3bce22402fff None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:22.083612 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353a772-609d-4008-b2a9-3bce22402fff None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:22.084879 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 61/121] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:22.088936 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad580a-c2fe-4d41-ace1-d972c75e0da9 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:22.090407 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353a772-609d-4008-b2a9-3bce22402fff None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:22.133078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e8044-f806-4598-aebc-b012350b73e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:22.133834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e8044-f806-4598-aebc-b012350b73e9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:22.134134 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e8044-f806-4598-aebc-b012350b73e9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:22.134441 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e8044-f806-4598-aebc-b012350b73e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:22.155583 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e8044-f806-4598-aebc-b012350b73e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:22.166439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-094e8044-f806-4598-aebc-b012350b73e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:48:22.166439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:22.166439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:22.166439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:22.166439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:22.166439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:22.166439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:22.166439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:22.166439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:22.166439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:22.166439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:22.166439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:22.166439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:22.166439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:22.166439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:22.166439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:22.167326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:22.167326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:22.167326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:22.167326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:22.167326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:22.167326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:22.167326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:22.167326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:22.167326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:22.167326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:22.167326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:22.167326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:48:22.167326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:22.167326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:22.167326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:22.167326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:22.168091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:22.168091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:22.168091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:22.168091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:22.168091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:22.168091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:22.168091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:22.168091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:22.168091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:22.168091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:22.168091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:48:22.168091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:48:22.168091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 61/122] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 9 22:48:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:22.169853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19ace0-51d1-4ecf-b0ba-b29bdbd90b04 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:22.170690 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19ace0-51d1-4ecf-b0ba-b29bdbd90b04 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:22.170690 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19ace0-51d1-4ecf-b0ba-b29bdbd90b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:22.191597 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-dd19ace0-51d1-4ecf-b0ba-b29bdbd90b04 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:22.193231 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-dd19ace0-51d1-4ecf-b0ba-b29bdbd90b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:22.211963 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353a772-609d-4008-b2a9-3bce22402fff None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:22.246264 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaeb202-9878-46c2-a79f-5c4cdcacead5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:22.247137 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaeb202-9878-46c2-a79f-5c4cdcacead5 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:22.247416 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaeb202-9878-46c2-a79f-5c4cdcacead5 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:22.247693 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaeb202-9878-46c2-a79f-5c4cdcacead5 None admin] PATH_INFO: `/v3/users/e627cb592eca48b482cb248bd3f1f207` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:22.261712 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaeb202-9878-46c2-a79f-5c4cdcacead5 None admin] RBAC: Authorizing `identity:get_user(user_id=e627cb592eca48b482cb248bd3f1f207)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:22.263561 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaeb202-9878-46c2-a79f-5c4cdcacead5 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:22.264462 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 62/123] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:22.265414 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 62/124] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:22 2022] GET /identity/v3/users/e627cb592eca48b482cb248bd3f1f207 => generated 289 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:22.265890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19ace0-51d1-4ecf-b0ba-b29bdbd90b04 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:22.266217 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaeb202-9878-46c2-a79f-5c4cdcacead5 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:22.364750 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaeb202-9878-46c2-a79f-5c4cdcacead5 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:22.366053 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19ace0-51d1-4ecf-b0ba-b29bdbd90b04 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:22.400217 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c195dc-3030-434c-ab9f-e2511e2abbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:22.400813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c195dc-3030-434c-ab9f-e2511e2abbaa None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:22.401093 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c195dc-3030-434c-ab9f-e2511e2abbaa None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:22.401373 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c195dc-3030-434c-ab9f-e2511e2abbaa None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:22.402150 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c195dc-3030-434c-ab9f-e2511e2abbaa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:22.402440 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c195dc-3030-434c-ab9f-e2511e2abbaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:22.404119 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c195dc-3030-434c-ab9f-e2511e2abbaa None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:22.406603 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24e17b-1ac9-4cab-93c8-da220cc62435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:22.407776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24e17b-1ac9-4cab-93c8-da220cc62435 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:22.408269 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24e17b-1ac9-4cab-93c8-da220cc62435 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:22.408749 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24e17b-1ac9-4cab-93c8-da220cc62435 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:22.409812 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24e17b-1ac9-4cab-93c8-da220cc62435 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:22.411800 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24e17b-1ac9-4cab-93c8-da220cc62435 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:22.417917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 63/125] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:48:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:22.419029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c195dc-3030-434c-ab9f-e2511e2abbaa None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:22.447202 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 63/126] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 9 22:48:22 2022] POST /identity/v3/projects => generated 302 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:48:22.448560 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24e17b-1ac9-4cab-93c8-da220cc62435 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:22.579923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24e17b-1ac9-4cab-93c8-da220cc62435 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:22.597186 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c195dc-3030-434c-ab9f-e2511e2abbaa None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:22.653883 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029ea8b-563e-4c45-bab2-3c12f59eab30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:22.654642 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029ea8b-563e-4c45-bab2-3c12f59eab30 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:22.654920 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029ea8b-563e-4c45-bab2-3c12f59eab30 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:22.655191 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029ea8b-563e-4c45-bab2-3c12f59eab30 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:22.666140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370c50f-33f3-4a51-a1be-78697d05eed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:22.666805 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370c50f-33f3-4a51-a1be-78697d05eed5 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:22.667091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370c50f-33f3-4a51-a1be-78697d05eed5 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:22.667364 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370c50f-33f3-4a51-a1be-78697d05eed5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:22.675973 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029ea8b-563e-4c45-bab2-3c12f59eab30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:22.690354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-3029ea8b-563e-4c45-bab2-3c12f59eab30 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:22.690354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:22.690354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:22.690354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:22.690354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:22.690354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:22.690354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:22.690354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:22.690354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:22.690354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:22.690354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:22.690354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:22.690354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:22.690354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:22.690354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:22.690354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:22.691311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:22.691311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:22.691311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:22.691311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:22.691311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:22.691311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:22.691311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:22.691311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:22.691311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:22.691311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:22.691311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:22.691311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:48:22.691311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:22.691311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:22.691311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:22.691311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:22.692097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:22.692097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:22.692097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:22.692097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:22.692097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:22.692097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:22.692097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:22.692097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:22.692097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:22.692097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:22.692097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:22.692097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:48:22.696302 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 64/127] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:22.696302 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029ea8b-563e-4c45-bab2-3c12f59eab30 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:22.757247 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9370c50f-33f3-4a51-a1be-78697d05eed5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:22.759412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9370c50f-33f3-4a51-a1be-78697d05eed5 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:22.821602 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029ea8b-563e-4c45-bab2-3c12f59eab30 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:22.870568 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3de868-f4b6-459a-b9d6-3ebc85a43140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:22.871721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3de868-f4b6-459a-b9d6-3ebc85a43140 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:22.872006 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3de868-f4b6-459a-b9d6-3ebc85a43140 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:22.872288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3de868-f4b6-459a-b9d6-3ebc85a43140 None admin] PATH_INFO: `/v3/users/11ec991028e64f3db9ef62861c931821` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:22.873521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3de868-f4b6-459a-b9d6-3ebc85a43140 None admin] RBAC: Authorizing `identity:get_user(user_id=11ec991028e64f3db9ef62861c931821)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:22.873708 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 64/128] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:48:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:48:22.874544 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370c50f-33f3-4a51-a1be-78697d05eed5 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:22.875411 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3de868-f4b6-459a-b9d6-3ebc85a43140 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:22.878712 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 65/129] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:22 2022] GET /identity/v3/users/11ec991028e64f3db9ef62861c931821 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:22.879512 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3de868-f4b6-459a-b9d6-3ebc85a43140 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:22.989997 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370c50f-33f3-4a51-a1be-78697d05eed5 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:23.035607 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3de868-f4b6-459a-b9d6-3ebc85a43140 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:23.045207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a66f4-577f-49e6-beb3-93e6d6f8bd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:23.045877 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a66f4-577f-49e6-beb3-93e6d6f8bd8d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:23.046174 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a66f4-577f-49e6-beb3-93e6d6f8bd8d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:23.046449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a66f4-577f-49e6-beb3-93e6d6f8bd8d None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:23.047192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a66f4-577f-49e6-beb3-93e6d6f8bd8d None admin] RBAC: Adding query filter params (name=member) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:23.047492 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a66f4-577f-49e6-beb3-93e6d6f8bd8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:23.048857 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a66f4-577f-49e6-beb3-93e6d6f8bd8d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:23.065957 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 65/130] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:23.066784 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a66f4-577f-49e6-beb3-93e6d6f8bd8d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:23.082418 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a58063-1aa7-4474-bee7-1c720d830083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:23.083649 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a58063-1aa7-4474-bee7-1c720d830083 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:23.084142 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a58063-1aa7-4474-bee7-1c720d830083 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:23.084623 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a58063-1aa7-4474-bee7-1c720d830083 None admin] PATH_INFO: `/v3/projects/1a6770d749f644eea4fd1584b1503d28` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:23.088065 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a58063-1aa7-4474-bee7-1c720d830083 None admin] RBAC: Authorizing `identity:get_project(project_id=1a6770d749f644eea4fd1584b1503d28)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:23.089767 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a58063-1aa7-4474-bee7-1c720d830083 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:23.091729 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 66/131] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:22 2022] GET /identity/v3/projects/1a6770d749f644eea4fd1584b1503d28 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:23.092713 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a58063-1aa7-4474-bee7-1c720d830083 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:23.198408 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a66f4-577f-49e6-beb3-93e6d6f8bd8d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:23.249595 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a58063-1aa7-4474-bee7-1c720d830083 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:23.262636 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ec892-af3e-4335-8ef3-1592b20f6dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:23.264213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ec892-af3e-4335-8ef3-1592b20f6dc3 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:23.264714 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ec892-af3e-4335-8ef3-1592b20f6dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:23.265209 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ec892-af3e-4335-8ef3-1592b20f6dc3 None admin] PATH_INFO: `/v3/projects/1a6770d749f644eea4fd1584b1503d28` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:23.266730 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ec892-af3e-4335-8ef3-1592b20f6dc3 None admin] RBAC: Authorizing `identity:get_project(project_id=1a6770d749f644eea4fd1584b1503d28)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:23.268476 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ec892-af3e-4335-8ef3-1592b20f6dc3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:23.278652 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 66/132] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:23 2022] GET /identity/v3/projects/1a6770d749f644eea4fd1584b1503d28 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:23.279973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ec892-af3e-4335-8ef3-1592b20f6dc3 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:23.297550 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8accce-4fba-4e34-909a-e76aa94a72e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:23.298444 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8accce-4fba-4e34-909a-e76aa94a72e1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:23.298722 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8accce-4fba-4e34-909a-e76aa94a72e1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:23.298991 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8accce-4fba-4e34-909a-e76aa94a72e1 None admin] PATH_INFO: `/v3/users/11ec991028e64f3db9ef62861c931821` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:23.300263 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8accce-4fba-4e34-909a-e76aa94a72e1 None admin] RBAC: Authorizing `identity:get_user(user_id=11ec991028e64f3db9ef62861c931821)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:23.303264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8accce-4fba-4e34-909a-e76aa94a72e1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:23.305806 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 67/133] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:23 2022] GET /identity/v3/users/11ec991028e64f3db9ef62861c931821 => generated 275 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:23.314272 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8accce-4fba-4e34-909a-e76aa94a72e1 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:23.397561 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ec892-af3e-4335-8ef3-1592b20f6dc3 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:23.405344 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8accce-4fba-4e34-909a-e76aa94a72e1 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:23.439284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095f160-4f5f-4480-acee-56db3f7a9248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:23.440271 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095f160-4f5f-4480-acee-56db3f7a9248 None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:23.440554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095f160-4f5f-4480-acee-56db3f7a9248 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:23.440834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095f160-4f5f-4480-acee-56db3f7a9248 None admin] PATH_INFO: `/v3/projects/1a6770d749f644eea4fd1584b1503d28/users/11ec991028e64f3db9ef62861c931821/roles/46ddc3cf4b124399a275bc639635e10a` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:23.443043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f7e66-92c1-4738-bfbf-2932eaec4454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:23.443689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f7e66-92c1-4738-bfbf-2932eaec4454 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:23.443963 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f7e66-92c1-4738-bfbf-2932eaec4454 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:23.444230 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f7e66-92c1-4738-bfbf-2932eaec4454 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:23.445085 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f7e66-92c1-4738-bfbf-2932eaec4454 None admin] RBAC: Adding query filter params (role.id=949d4917fcdb41e2a938aadb277d1970, scope.project.id=1a6770d749f644eea4fd1584b1503d28, user.id=e627cb592eca48b482cb248bd3f1f207) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:23.445391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f7e66-92c1-4738-bfbf-2932eaec4454 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:23.447327 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4095f160-4f5f-4480-acee-56db3f7a9248 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a6770d749f644eea4fd1584b1503d28, user_id=11ec991028e64f3db9ef62861c931821, role_id=46ddc3cf4b124399a275bc639635e10a)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:23.448764 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4095f160-4f5f-4480-acee-56db3f7a9248 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:23.451823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f7e66-92c1-4738-bfbf-2932eaec4454 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:23.460926 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 67/134] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:48:23 2022] GET /identity/v3/role_assignments?user.id=e627cb592eca48b482cb248bd3f1f207&scope.project.id=1a6770d749f644eea4fd1584b1503d28&role.id=949d4917fcdb41e2a938aadb277d1970 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:23.461772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f7e66-92c1-4738-bfbf-2932eaec4454 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:23.494827 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 68/135] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:48:23 2022] PUT /identity/v3/projects/1a6770d749f644eea4fd1584b1503d28/users/11ec991028e64f3db9ef62861c931821/roles/46ddc3cf4b124399a275bc639635e10a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:48:23.589168 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f7e66-92c1-4738-bfbf-2932eaec4454 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:23.633500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02761b5-0dd8-497f-9a87-84ee6453dd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:23.636771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02761b5-0dd8-497f-9a87-84ee6453dd8c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:23.636771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02761b5-0dd8-497f-9a87-84ee6453dd8c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:23.636771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02761b5-0dd8-497f-9a87-84ee6453dd8c None admin] PATH_INFO: `/v3/projects/f8933570c9dc44c8bb1ecbd74374889b` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:23.636771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02761b5-0dd8-497f-9a87-84ee6453dd8c None admin] RBAC: Authorizing `identity:get_project(project_id=f8933570c9dc44c8bb1ecbd74374889b)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:23.637673 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02761b5-0dd8-497f-9a87-84ee6453dd8c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:23.639408 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 68/136] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:23 2022] GET /identity/v3/projects/f8933570c9dc44c8bb1ecbd74374889b => generated 312 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:23.647268 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095f160-4f5f-4480-acee-56db3f7a9248 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:23.795602 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095f160-4f5f-4480-acee-56db3f7a9248 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:23.831181 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54c7b8-eb14-4de4-997a-d76f01a29e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:23.832070 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54c7b8-eb14-4de4-997a-d76f01a29e71 None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:23.832351 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54c7b8-eb14-4de4-997a-d76f01a29e71 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:23.832632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54c7b8-eb14-4de4-997a-d76f01a29e71 None admin] PATH_INFO: `/v3/projects/f8933570c9dc44c8bb1ecbd74374889b/users/11ec991028e64f3db9ef62861c931821/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:23.834734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c54c7b8-eb14-4de4-997a-d76f01a29e71 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8933570c9dc44c8bb1ecbd74374889b, user_id=11ec991028e64f3db9ef62861c931821, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:23.836835 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c54c7b8-eb14-4de4-997a-d76f01a29e71 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:23.862431 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 69/137] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:48:23 2022] PUT /identity/v3/projects/f8933570c9dc44c8bb1ecbd74374889b/users/11ec991028e64f3db9ef62861c931821/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:48:24.400575 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54a702-a3d4-442a-b453-61ebcea15c08 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:24.401120 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54a702-a3d4-442a-b453-61ebcea15c08 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:24.401120 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54a702-a3d4-442a-b453-61ebcea15c08 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:24.402533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 69/138] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:24.408072 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0325c-c83a-4044-b5a5-d7920be3a8ec None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:24.408526 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0325c-c83a-4044-b5a5-d7920be3a8ec None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:24.408616 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0325c-c83a-4044-b5a5-d7920be3a8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:24.425647 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-d0b0325c-c83a-4044-b5a5-d7920be3a8ec None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:24.426882 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d0b0325c-c83a-4044-b5a5-d7920be3a8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:24.498260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 70/139] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:24.521198 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54a702-a3d4-442a-b453-61ebcea15c08 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:24.625095 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54a702-a3d4-442a-b453-61ebcea15c08 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:24.656645 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481a439-6ba7-4f70-8e54-c9c740ef7f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:24.657361 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481a439-6ba7-4f70-8e54-c9c740ef7f0a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:24.657639 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481a439-6ba7-4f70-8e54-c9c740ef7f0a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:24.657908 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481a439-6ba7-4f70-8e54-c9c740ef7f0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:24.668332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481a439-6ba7-4f70-8e54-c9c740ef7f0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:24.675898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-f481a439-6ba7-4f70-8e54-c9c740ef7f0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:48:24.675898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:24.675898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:24.675898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:24.675898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:24.675898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:24.675898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:24.675898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:24.675898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:24.675898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:24.675898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:24.675898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:24.675898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:24.675898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:24.675898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:24.675898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:24.676813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:24.676813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:24.676813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:24.676813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:24.676813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:24.676813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:24.676813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:24.676813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:24.676813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:24.676813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:24.676813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:24.676813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:24.676813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:24.676813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:24.676813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:24.676813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:24.677631 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:24.677631 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:24.677631 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:24.677631 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:24.677631 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:24.677631 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:24.677631 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:24.677631 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:24.677631 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:24.677631 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:24.677631 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:48:24.677631 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:24.682190 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 70/140] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:48:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:24.683763 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0325c-c83a-4044-b5a5-d7920be3a8ec None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:24.708750 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3cff8-8c77-422e-9246-2ad7eca977ea None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:24.709059 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3cff8-8c77-422e-9246-2ad7eca977ea None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:24.709305 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3cff8-8c77-422e-9246-2ad7eca977ea None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:24.710764 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 71/141] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:48:24.716027 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116e2ed-e0e1-4bcf-953b-89e81fb709d0 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:24.716339 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116e2ed-e0e1-4bcf-953b-89e81fb709d0 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:24.716591 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116e2ed-e0e1-4bcf-953b-89e81fb709d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:24.744439 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-1116e2ed-e0e1-4bcf-953b-89e81fb709d0 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:24.745634 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1116e2ed-e0e1-4bcf-953b-89e81fb709d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:24.831366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 72/142] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:24.857198 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116e2ed-e0e1-4bcf-953b-89e81fb709d0 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:24.867550 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0325c-c83a-4044-b5a5-d7920be3a8ec None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:24.929238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fccb7-1e68-4721-b86e-b4a2d7102e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:24.930338 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fccb7-1e68-4721-b86e-b4a2d7102e50 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:24.930628 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fccb7-1e68-4721-b86e-b4a2d7102e50 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:24.930903 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fccb7-1e68-4721-b86e-b4a2d7102e50 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:24.931654 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45fccb7-1e68-4721-b86e-b4a2d7102e50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:24.931957 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45fccb7-1e68-4721-b86e-b4a2d7102e50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:24.933368 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45fccb7-1e68-4721-b86e-b4a2d7102e50 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:24.951204 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 71/143] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:24.957525 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fccb7-1e68-4721-b86e-b4a2d7102e50 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:25.025523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116e2ed-e0e1-4bcf-953b-89e81fb709d0 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:25.099198 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc36c3c-b742-41bd-94d0-2ad4b7c29f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:25.100255 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc36c3c-b742-41bd-94d0-2ad4b7c29f6b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:25.100693 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc36c3c-b742-41bd-94d0-2ad4b7c29f6b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:25.101632 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc36c3c-b742-41bd-94d0-2ad4b7c29f6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:25.123035 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc36c3c-b742-41bd-94d0-2ad4b7c29f6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:25.130834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-0cc36c3c-b742-41bd-94d0-2ad4b7c29f6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:25.130834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:25.130834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:25.130834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:25.130834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:25.130834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:25.130834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:25.130834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:25.130834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:25.130834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:25.130834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:25.130834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:25.130834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:25.130834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:25.130834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:25.130834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:25.131703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:25.131703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:25.131703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:25.131703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:25.131703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:25.131703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:25.131703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:25.131703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:25.131703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:25.131703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:25.131703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:25.131703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:25.131703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:25.131703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:25.131703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:25.131703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:25.132501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:25.132501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:25.132501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:25.132501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:25.132501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:25.132501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:25.132501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:25.132501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:25.132501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:25.132501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:25.132501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:25.132501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:25.139486 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 73/144] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:25.140931 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc36c3c-b742-41bd-94d0-2ad4b7c29f6b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:25.142767 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fccb7-1e68-4721-b86e-b4a2d7102e50 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:25.223455 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada96459-ce5e-49a2-ac81-be985d9c584d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:25.227248 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada96459-ce5e-49a2-ac81-be985d9c584d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:25.228429 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada96459-ce5e-49a2-ac81-be985d9c584d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:25.228921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada96459-ce5e-49a2-ac81-be985d9c584d None admin] PATH_INFO: `/v3/users/873ba60d483a41a4b36c04c1aa8b34ad` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:25.232061 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada96459-ce5e-49a2-ac81-be985d9c584d None admin] RBAC: Authorizing `identity:get_user(user_id=873ba60d483a41a4b36c04c1aa8b34ad)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:25.236058 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada96459-ce5e-49a2-ac81-be985d9c584d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:25.246140 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 72/145] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:24 2022] GET /identity/v3/users/873ba60d483a41a4b36c04c1aa8b34ad => generated 247 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:25.247409 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada96459-ce5e-49a2-ac81-be985d9c584d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:25.381778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc36c3c-b742-41bd-94d0-2ad4b7c29f6b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:25.450492 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada96459-ce5e-49a2-ac81-be985d9c584d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:25.455471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7da217-0d8b-4054-b379-70ca07a8e6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:25.456156 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7da217-0d8b-4054-b379-70ca07a8e6d2 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:25.456444 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7da217-0d8b-4054-b379-70ca07a8e6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:25.456723 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7da217-0d8b-4054-b379-70ca07a8e6d2 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:25.457480 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7da217-0d8b-4054-b379-70ca07a8e6d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:25.457791 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7da217-0d8b-4054-b379-70ca07a8e6d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:25.459283 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7da217-0d8b-4054-b379-70ca07a8e6d2 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:25.478863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 74/146] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:25.497290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7da217-0d8b-4054-b379-70ca07a8e6d2 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:25.510812 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3da2b-482c-478c-89c8-02ef1615eef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:25.511775 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3da2b-482c-478c-89c8-02ef1615eef9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:25.512250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3da2b-482c-478c-89c8-02ef1615eef9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:25.512734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3da2b-482c-478c-89c8-02ef1615eef9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:25.594797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3da2b-482c-478c-89c8-02ef1615eef9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:25.596604 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3da2b-482c-478c-89c8-02ef1615eef9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:25.699404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 73/147] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:48:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:48:25.707287 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3da2b-482c-478c-89c8-02ef1615eef9 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:25.760320 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7da217-0d8b-4054-b379-70ca07a8e6d2 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:25.854588 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3da2b-482c-478c-89c8-02ef1615eef9 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:25.858856 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeafa0d-259e-457b-ab12-4a0c589f685d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:25.860022 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeafa0d-259e-457b-ab12-4a0c589f685d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:25.860311 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeafa0d-259e-457b-ab12-4a0c589f685d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:25.860587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeafa0d-259e-457b-ab12-4a0c589f685d None admin] PATH_INFO: `/v3/users/11ec991028e64f3db9ef62861c931821` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:25.861902 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeafa0d-259e-457b-ab12-4a0c589f685d None admin] RBAC: Authorizing `identity:get_user(user_id=11ec991028e64f3db9ef62861c931821)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:25.863371 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeafa0d-259e-457b-ab12-4a0c589f685d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:25.865000 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 75/148] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:25 2022] GET /identity/v3/users/11ec991028e64f3db9ef62861c931821 => generated 275 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:25.875747 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622513a2-c3d2-4900-8f7a-f0b98775b4a1 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:25.876035 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622513a2-c3d2-4900-8f7a-f0b98775b4a1 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:25.876325 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622513a2-c3d2-4900-8f7a-f0b98775b4a1 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:25.877903 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 76/149] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:25.883118 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554b99f-22e5-4ccf-9eba-38b2af03bac4 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:25.883377 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554b99f-22e5-4ccf-9eba-38b2af03bac4 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:25.883657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554b99f-22e5-4ccf-9eba-38b2af03bac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:25.900179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093f8a6-135d-45a9-9abf-06b5827d33d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:25.901041 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093f8a6-135d-45a9-9abf-06b5827d33d5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:25.901325 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093f8a6-135d-45a9-9abf-06b5827d33d5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:25.901602 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093f8a6-135d-45a9-9abf-06b5827d33d5 None admin] PATH_INFO: `/v3/projects/1a6770d749f644eea4fd1584b1503d28` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:25.907831 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093f8a6-135d-45a9-9abf-06b5827d33d5 None admin] RBAC: Authorizing `identity:get_project(project_id=1a6770d749f644eea4fd1584b1503d28)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:25.909288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093f8a6-135d-45a9-9abf-06b5827d33d5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:25.911225 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 74/150] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:25 2022] GET /identity/v3/projects/1a6770d749f644eea4fd1584b1503d28 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:25.911828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093f8a6-135d-45a9-9abf-06b5827d33d5 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:25.919643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a554b99f-22e5-4ccf-9eba-38b2af03bac4 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:25.920814 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a554b99f-22e5-4ccf-9eba-38b2af03bac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:26.017794 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093f8a6-135d-45a9-9abf-06b5827d33d5 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:26.036742 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 77/151] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 09 22:48:26.038019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554b99f-22e5-4ccf-9eba-38b2af03bac4 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:26.078162 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83c9f1-fc4d-462a-a2cd-504d2f8df17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:26.078162 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83c9f1-fc4d-462a-a2cd-504d2f8df17b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:26.078162 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83c9f1-fc4d-462a-a2cd-504d2f8df17b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:26.078162 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83c9f1-fc4d-462a-a2cd-504d2f8df17b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:26.142001 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae83c9f1-fc4d-462a-a2cd-504d2f8df17b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:26.143170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae83c9f1-fc4d-462a-a2cd-504d2f8df17b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:26.171493 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554b99f-22e5-4ccf-9eba-38b2af03bac4 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:26.213747 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd913aa2-f499-403b-a88a-185dea35358d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:26.214802 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd913aa2-f499-403b-a88a-185dea35358d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:26.215281 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd913aa2-f499-403b-a88a-185dea35358d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:26.215761 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd913aa2-f499-403b-a88a-185dea35358d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:26.216868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd913aa2-f499-403b-a88a-185dea35358d None admin] RBAC: Adding query filter params (role.id=bda550492f4f4c5c83f52fa89fc18a7d, scope.project.id=1a6770d749f644eea4fd1584b1503d28, user.id=873ba60d483a41a4b36c04c1aa8b34ad) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:26.217377 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd913aa2-f499-403b-a88a-185dea35358d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:26.219146 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd913aa2-f499-403b-a88a-185dea35358d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:26.233578 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 78/152] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:48:26 2022] GET /identity/v3/role_assignments?user.id=873ba60d483a41a4b36c04c1aa8b34ad&scope.project.id=1a6770d749f644eea4fd1584b1503d28&role.id=bda550492f4f4c5c83f52fa89fc18a7d => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:26.235034 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd913aa2-f499-403b-a88a-185dea35358d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:26.246160 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 75/153] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:48:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:48:26.255218 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83c9f1-fc4d-462a-a2cd-504d2f8df17b None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:26.352011 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd913aa2-f499-403b-a88a-185dea35358d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:26.361169 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83c9f1-fc4d-462a-a2cd-504d2f8df17b None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:26.396649 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def417d7-6c37-4813-afb1-c67671643e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:26.397465 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def417d7-6c37-4813-afb1-c67671643e53 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:26.397740 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def417d7-6c37-4813-afb1-c67671643e53 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:26.400118 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def417d7-6c37-4813-afb1-c67671643e53 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:26.401590 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def417d7-6c37-4813-afb1-c67671643e53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:26.403089 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def417d7-6c37-4813-afb1-c67671643e53 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:26.406108 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97678ad-a1e9-439c-a357-1086f7b28772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:26.406302 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 79/154] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:48:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:26.407232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97678ad-a1e9-439c-a357-1086f7b28772 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:26.407522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97678ad-a1e9-439c-a357-1086f7b28772 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:26.407799 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97678ad-a1e9-439c-a357-1086f7b28772 None admin] PATH_INFO: `/v3/projects/1a6770d749f644eea4fd1584b1503d28` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:26.409000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97678ad-a1e9-439c-a357-1086f7b28772 None admin] RBAC: Authorizing `identity:get_project(project_id=1a6770d749f644eea4fd1584b1503d28)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:26.410567 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97678ad-a1e9-439c-a357-1086f7b28772 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:26.412666 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 76/155] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:26 2022] GET /identity/v3/projects/1a6770d749f644eea4fd1584b1503d28 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:26.413472 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def417d7-6c37-4813-afb1-c67671643e53 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:26.418575 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97678ad-a1e9-439c-a357-1086f7b28772 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:26.600145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97678ad-a1e9-439c-a357-1086f7b28772 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:26.605158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def417d7-6c37-4813-afb1-c67671643e53 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:26.643100 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b48ae-5d96-4e0a-be73-70d009d9ff0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:26.643776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b48ae-5d96-4e0a-be73-70d009d9ff0f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:26.644062 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b48ae-5d96-4e0a-be73-70d009d9ff0f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:26.644402 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b48ae-5d96-4e0a-be73-70d009d9ff0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:26.645180 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b48ae-5d96-4e0a-be73-70d009d9ff0f None admin] RBAC: Adding query filter params (role.id=3631750e6bf34730aa3557cd90aa6108, scope.project.id=1a6770d749f644eea4fd1584b1503d28, user.id=11ec991028e64f3db9ef62861c931821) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:26.645498 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b48ae-5d96-4e0a-be73-70d009d9ff0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:26.646980 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b48ae-5d96-4e0a-be73-70d009d9ff0f None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:26.654180 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 77/156] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:48:26 2022] GET /identity/v3/role_assignments?user.id=11ec991028e64f3db9ef62861c931821&scope.project.id=1a6770d749f644eea4fd1584b1503d28&role.id=3631750e6bf34730aa3557cd90aa6108 => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:26.692425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dcd1cf-25ec-4432-b7ad-d9a3be52051e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:26.693110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcd1cf-25ec-4432-b7ad-d9a3be52051e None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:26.693384 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcd1cf-25ec-4432-b7ad-d9a3be52051e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:26.693656 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcd1cf-25ec-4432-b7ad-d9a3be52051e None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:26.694436 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dcd1cf-25ec-4432-b7ad-d9a3be52051e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:26.696573 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dcd1cf-25ec-4432-b7ad-d9a3be52051e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:26.735106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 80/157] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 9 22:48:26 2022] POST /identity/v3/users => generated 283 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:48:27.063696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce2feb-0681-47f4-8b79-79a32d2f0b46 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:27.064079 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce2feb-0681-47f4-8b79-79a32d2f0b46 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:27.064264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce2feb-0681-47f4-8b79-79a32d2f0b46 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:27.065891 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 78/158] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:27.071665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed08efd-4156-45bf-a37d-23ed5ee7c8bc None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:27.071953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed08efd-4156-45bf-a37d-23ed5ee7c8bc None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:27.072231 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed08efd-4156-45bf-a37d-23ed5ee7c8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:27.091528 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-4ed08efd-4156-45bf-a37d-23ed5ee7c8bc None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:27.092781 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-4ed08efd-4156-45bf-a37d-23ed5ee7c8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:27.099071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e7c48-0e94-4f82-8535-e4ade7b4060c None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:27.099355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e7c48-0e94-4f82-8535-e4ade7b4060c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:27.099657 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e7c48-0e94-4f82-8535-e4ade7b4060c None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:27.101834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 79/159] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:27.106724 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd498634-3625-4540-8682-a8d931a909cd None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:27.107021 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd498634-3625-4540-8682-a8d931a909cd None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:27.107300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd498634-3625-4540-8682-a8d931a909cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:27.127167 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-cd498634-3625-4540-8682-a8d931a909cd None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:27.128356 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-cd498634-3625-4540-8682-a8d931a909cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:27.187611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 81/160] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:27.188517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f552713-5530-4571-a14b-a20f0865c407 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:27.188811 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f552713-5530-4571-a14b-a20f0865c407 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:27.189085 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f552713-5530-4571-a14b-a20f0865c407 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:27.190648 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 82/161] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:27.195795 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14b38b-7631-4c22-9e0d-59a0c1aa4e82 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:27.196103 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14b38b-7631-4c22-9e0d-59a0c1aa4e82 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:27.196353 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14b38b-7631-4c22-9e0d-59a0c1aa4e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:27.211304 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 80/162] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:48:27.212031 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd498634-3625-4540-8682-a8d931a909cd None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:27.213488 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-4b14b38b-7631-4c22-9e0d-59a0c1aa4e82 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:27.214758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-4b14b38b-7631-4c22-9e0d-59a0c1aa4e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:27.298070 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 83/163] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:48:27.298779 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14b38b-7631-4c22-9e0d-59a0c1aa4e82 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:27.337048 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd498634-3625-4540-8682-a8d931a909cd None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:27.381844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff28db4-f63c-45f7-91ed-29a430507c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:27.383193 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff28db4-f63c-45f7-91ed-29a430507c46 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:27.383485 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff28db4-f63c-45f7-91ed-29a430507c46 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:27.383758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff28db4-f63c-45f7-91ed-29a430507c46 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:27.395952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff28db4-f63c-45f7-91ed-29a430507c46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:27.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-1ff28db4-f63c-45f7-91ed-29a430507c46 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:48:27.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:27.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:27.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:27.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:27.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:27.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:27.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:27.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:27.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:27.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:27.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:27.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:27.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:27.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:27.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:27.404965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:27.404965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:27.404965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:27.404965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:27.404965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:27.404965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:27.404965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:27.404965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:27.404965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:27.404965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:27.404965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:27.404965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:48:27.404965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:27.404965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:27.404965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:27.404965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:27.406288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:27.406288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:27.406288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:27.406288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:27.406288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:27.406288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:27.406288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:27.406288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:27.406288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:27.406288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:27.406288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:48:27.406288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:48:27.406288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 81/164] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 9 22:48:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:27.406288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff28db4-f63c-45f7-91ed-29a430507c46 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:27.433597 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14b38b-7631-4c22-9e0d-59a0c1aa4e82 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:27.467991 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e69ed1-a8c6-4617-838b-9a1c34f79211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:27.469136 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e69ed1-a8c6-4617-838b-9a1c34f79211 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:27.469652 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e69ed1-a8c6-4617-838b-9a1c34f79211 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:27.469921 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e69ed1-a8c6-4617-838b-9a1c34f79211 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:27.480977 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e69ed1-a8c6-4617-838b-9a1c34f79211 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:27.489004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-00e69ed1-a8c6-4617-838b-9a1c34f79211 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 22:48:27.489004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:27.489004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:27.489004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:27.489004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:27.489004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:27.489004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:27.489004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:27.489004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:27.489004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:27.489004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:27.489004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:27.489004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:27.489004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:27.489004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:27.489004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:27.489999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:27.489999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:27.489999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:27.489999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:27.489999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:27.489999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:27.489999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:27.489999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:27.489999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:27.489999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:27.489999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:27.489999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:27.489999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:27.489999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:27.489999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:27.489999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:27.490949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:27.490949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:27.490949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:27.490949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:27.490949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:27.490949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:27.490949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:27.490949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:27.490949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:27.490949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:27.490949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 22:48:27.490949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:27.494220 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 84/165] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:27.495061 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e69ed1-a8c6-4617-838b-9a1c34f79211 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:27.523795 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff28db4-f63c-45f7-91ed-29a430507c46 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:27.558249 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d0cc4-e45a-4286-a2d3-3b86309eecfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:27.559319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d0cc4-e45a-4286-a2d3-3b86309eecfb None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:27.559745 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d0cc4-e45a-4286-a2d3-3b86309eecfb None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:27.560161 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d0cc4-e45a-4286-a2d3-3b86309eecfb None admin] PATH_INFO: `/v3/roles/member` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:27.585186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d0cc4-e45a-4286-a2d3-3b86309eecfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:27.598278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-ec2d0cc4-e45a-4286-a2d3-3b86309eecfb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:27.598278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:27.598278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:27.598278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:27.598278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:27.598278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:27.598278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:27.598278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:27.598278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:27.598278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:27.598278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:27.598278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:27.598278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:27.598278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:27.598278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:27.598278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:27.599440 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:27.599440 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:27.599440 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:27.599440 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:27.599440 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:27.599440 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:27.599440 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:27.599440 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:27.599440 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:27.599440 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:27.599440 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:27.599440 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:48:27.599440 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:27.599440 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:27.599440 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:27.599440 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:27.600771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:27.600771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:27.600771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:27.600771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:27.600771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:27.600771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:27.600771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:27.600771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:27.600771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:27.600771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:27.600771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:27.600771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:48:27.601953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e69ed1-a8c6-4617-838b-9a1c34f79211 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:27.603553 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 82/166] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:27.609506 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d0cc4-e45a-4286-a2d3-3b86309eecfb None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:27.637277 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b38b37-d5bd-4fd4-ad3d-0bc163dfcfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:27.638593 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b38b37-d5bd-4fd4-ad3d-0bc163dfcfd2 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:27.639106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b38b37-d5bd-4fd4-ad3d-0bc163dfcfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:27.639567 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b38b37-d5bd-4fd4-ad3d-0bc163dfcfd2 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:27.640734 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b38b37-d5bd-4fd4-ad3d-0bc163dfcfd2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:27.642388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b38b37-d5bd-4fd4-ad3d-0bc163dfcfd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:27.644952 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b38b37-d5bd-4fd4-ad3d-0bc163dfcfd2 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:27.658674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 85/167] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:48:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:27.661111 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b38b37-d5bd-4fd4-ad3d-0bc163dfcfd2 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:27.793553 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b38b37-d5bd-4fd4-ad3d-0bc163dfcfd2 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:27.816358 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d0cc4-e45a-4286-a2d3-3b86309eecfb None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:27.828007 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ab556-5b7d-4345-9703-ff64135e013a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:27.829012 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ab556-5b7d-4345-9703-ff64135e013a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:27.829484 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ab556-5b7d-4345-9703-ff64135e013a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:27.829947 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ab556-5b7d-4345-9703-ff64135e013a None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:27.830896 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ab556-5b7d-4345-9703-ff64135e013a None admin] RBAC: Adding query filter params (name=member) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:27.831440 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ab556-5b7d-4345-9703-ff64135e013a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:27.833056 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ab556-5b7d-4345-9703-ff64135e013a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:27.842217 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 86/168] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:27.843437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ab556-5b7d-4345-9703-ff64135e013a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:27.909798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb98b5-0e43-4fff-a3ed-ce7da0b00f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:27.911707 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb98b5-0e43-4fff-a3ed-ce7da0b00f4a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:27.914587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb98b5-0e43-4fff-a3ed-ce7da0b00f4a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:27.915069 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb98b5-0e43-4fff-a3ed-ce7da0b00f4a None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:27.916017 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb98b5-0e43-4fff-a3ed-ce7da0b00f4a None admin] RBAC: Adding query filter params (name=reader) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:27.916511 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb98b5-0e43-4fff-a3ed-ce7da0b00f4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:27.919346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb98b5-0e43-4fff-a3ed-ce7da0b00f4a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:27.938903 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 83/169] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:27.942222 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb98b5-0e43-4fff-a3ed-ce7da0b00f4a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:28.014902 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ab556-5b7d-4345-9703-ff64135e013a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:28.085226 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2939fb97-b16c-4cfc-bb3b-18c2d73d6c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:28.087086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939fb97-b16c-4cfc-bb3b-18c2d73d6c6a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:28.087575 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939fb97-b16c-4cfc-bb3b-18c2d73d6c6a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:28.088037 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939fb97-b16c-4cfc-bb3b-18c2d73d6c6a None admin] PATH_INFO: `/v3/users/11ec991028e64f3db9ef62861c931821` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:28.089418 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2939fb97-b16c-4cfc-bb3b-18c2d73d6c6a None admin] RBAC: Authorizing `identity:get_user(user_id=11ec991028e64f3db9ef62861c931821)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:28.092528 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2939fb97-b16c-4cfc-bb3b-18c2d73d6c6a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:28.095109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 87/170] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:27 2022] GET /identity/v3/users/11ec991028e64f3db9ef62861c931821 => generated 275 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:28.097752 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2939fb97-b16c-4cfc-bb3b-18c2d73d6c6a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:28.144252 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb98b5-0e43-4fff-a3ed-ce7da0b00f4a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:28.206667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2939fb97-b16c-4cfc-bb3b-18c2d73d6c6a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:28.235085 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4b160-b78f-4a34-be71-698742ba9905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:28.235956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4b160-b78f-4a34-be71-698742ba9905 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:28.236450 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4b160-b78f-4a34-be71-698742ba9905 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:28.236555 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4b160-b78f-4a34-be71-698742ba9905 None admin] PATH_INFO: `/v3/users/11ec991028e64f3db9ef62861c931821` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:28.238720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4b160-b78f-4a34-be71-698742ba9905 None admin] RBAC: Authorizing `identity:get_user(user_id=11ec991028e64f3db9ef62861c931821)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:28.241251 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4b160-b78f-4a34-be71-698742ba9905 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:28.244581 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 84/171] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:27 2022] GET /identity/v3/users/11ec991028e64f3db9ef62861c931821 => generated 275 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:28.246649 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577d139-360e-4f2e-ae47-8956a979facf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:28.247105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4b160-b78f-4a34-be71-698742ba9905 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:28.248129 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577d139-360e-4f2e-ae47-8956a979facf None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:28.248596 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577d139-360e-4f2e-ae47-8956a979facf None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:28.249184 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577d139-360e-4f2e-ae47-8956a979facf None admin] PATH_INFO: `/v3/users/e627cb592eca48b482cb248bd3f1f207` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:28.250600 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f577d139-360e-4f2e-ae47-8956a979facf None admin] RBAC: Authorizing `identity:get_user(user_id=e627cb592eca48b482cb248bd3f1f207)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:28.252268 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f577d139-360e-4f2e-ae47-8956a979facf None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:28.254215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 88/172] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:28 2022] GET /identity/v3/users/e627cb592eca48b482cb248bd3f1f207 => generated 289 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:28.255279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577d139-360e-4f2e-ae47-8956a979facf None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:28.359350 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577d139-360e-4f2e-ae47-8956a979facf None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:28.398029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e365b90-cc6c-4692-8b49-27fc3ea18c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:28.399737 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e365b90-cc6c-4692-8b49-27fc3ea18c9b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:28.400861 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e365b90-cc6c-4692-8b49-27fc3ea18c9b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:28.401506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e365b90-cc6c-4692-8b49-27fc3ea18c9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:28.478454 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4b160-b78f-4a34-be71-698742ba9905 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:28.527953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e365b90-cc6c-4692-8b49-27fc3ea18c9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:28.531422 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e365b90-cc6c-4692-8b49-27fc3ea18c9b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:28.547018 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e798b0-a8e0-4d6b-bc67-ac5123e16f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:28.548065 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e798b0-a8e0-4d6b-bc67-ac5123e16f78 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:28.548538 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e798b0-a8e0-4d6b-bc67-ac5123e16f78 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:28.549003 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e798b0-a8e0-4d6b-bc67-ac5123e16f78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:28.677041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 89/173] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:48:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:48:28.677958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e365b90-cc6c-4692-8b49-27fc3ea18c9b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:28.707208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e798b0-a8e0-4d6b-bc67-ac5123e16f78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:28.709350 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e798b0-a8e0-4d6b-bc67-ac5123e16f78 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:28.774506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e365b90-cc6c-4692-8b49-27fc3ea18c9b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:28.811676 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcefc7-ef2c-46db-bce6-8b5059e06a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:28.812524 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcefc7-ef2c-46db-bce6-8b5059e06a14 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:28.813011 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcefc7-ef2c-46db-bce6-8b5059e06a14 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:28.813280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcefc7-ef2c-46db-bce6-8b5059e06a14 None admin] PATH_INFO: `/v3/projects/1a6770d749f644eea4fd1584b1503d28` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:28.815830 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bcefc7-ef2c-46db-bce6-8b5059e06a14 None admin] RBAC: Authorizing `identity:get_project(project_id=1a6770d749f644eea4fd1584b1503d28)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:28.817266 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bcefc7-ef2c-46db-bce6-8b5059e06a14 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:28.818970 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 90/174] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:28 2022] GET /identity/v3/projects/1a6770d749f644eea4fd1584b1503d28 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:28.821423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcefc7-ef2c-46db-bce6-8b5059e06a14 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:28.934868 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 85/175] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:48:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 689 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:48:28.935713 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e798b0-a8e0-4d6b-bc67-ac5123e16f78 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:28.948058 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcefc7-ef2c-46db-bce6-8b5059e06a14 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:28.983500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dacdbb-4cb9-45d9-a222-1856dcc42e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:28.984365 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dacdbb-4cb9-45d9-a222-1856dcc42e14 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:28.984914 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dacdbb-4cb9-45d9-a222-1856dcc42e14 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:28.985187 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dacdbb-4cb9-45d9-a222-1856dcc42e14 None admin] PATH_INFO: `/v3/projects/f8933570c9dc44c8bb1ecbd74374889b` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:28.992328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dacdbb-4cb9-45d9-a222-1856dcc42e14 None admin] RBAC: Authorizing `identity:get_project(project_id=f8933570c9dc44c8bb1ecbd74374889b)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:28.992328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dacdbb-4cb9-45d9-a222-1856dcc42e14 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:28.994902 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 91/176] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:28 2022] GET /identity/v3/projects/f8933570c9dc44c8bb1ecbd74374889b => generated 312 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:28.994902 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dacdbb-4cb9-45d9-a222-1856dcc42e14 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:29.143368 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e798b0-a8e0-4d6b-bc67-ac5123e16f78 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:29.160991 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dacdbb-4cb9-45d9-a222-1856dcc42e14 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:29.186487 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b875801-e5e8-42c8-ac39-4e80bf776115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:29.187758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b875801-e5e8-42c8-ac39-4e80bf776115 None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:29.188239 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b875801-e5e8-42c8-ac39-4e80bf776115 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:29.188729 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b875801-e5e8-42c8-ac39-4e80bf776115 None admin] PATH_INFO: `/v3/projects/1a6770d749f644eea4fd1584b1503d28/users/e627cb592eca48b482cb248bd3f1f207/roles/949d4917fcdb41e2a938aadb277d1970` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:29.192328 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b875801-e5e8-42c8-ac39-4e80bf776115 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a6770d749f644eea4fd1584b1503d28, user_id=e627cb592eca48b482cb248bd3f1f207, role_id=949d4917fcdb41e2a938aadb277d1970)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:29.194022 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b875801-e5e8-42c8-ac39-4e80bf776115 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:29.212059 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74411fc-0352-4e99-9055-18396e70206c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:29.213253 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74411fc-0352-4e99-9055-18396e70206c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:29.213753 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74411fc-0352-4e99-9055-18396e70206c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:29.214256 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74411fc-0352-4e99-9055-18396e70206c None admin] PATH_INFO: `/v3/projects/1a6770d749f644eea4fd1584b1503d28` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:29.221961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 86/177] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:48:28 2022] PUT /identity/v3/projects/1a6770d749f644eea4fd1584b1503d28/users/e627cb592eca48b482cb248bd3f1f207/roles/949d4917fcdb41e2a938aadb277d1970 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:48:29.223065 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b875801-e5e8-42c8-ac39-4e80bf776115 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:29.224027 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74411fc-0352-4e99-9055-18396e70206c None admin] RBAC: Authorizing `identity:get_project(project_id=1a6770d749f644eea4fd1584b1503d28)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:29.229906 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74411fc-0352-4e99-9055-18396e70206c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:29.237713 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 92/178] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:28 2022] GET /identity/v3/projects/1a6770d749f644eea4fd1584b1503d28 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:29.244926 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74411fc-0352-4e99-9055-18396e70206c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:29.364800 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b875801-e5e8-42c8-ac39-4e80bf776115 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:29.405054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e61f3-867d-48c5-84a8-b1c2f460757b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:29.406145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e61f3-867d-48c5-84a8-b1c2f460757b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:29.406605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e61f3-867d-48c5-84a8-b1c2f460757b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:29.407051 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e61f3-867d-48c5-84a8-b1c2f460757b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:29.408232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e61f3-867d-48c5-84a8-b1c2f460757b None admin] RBAC: Adding query filter params (role.id=3631750e6bf34730aa3557cd90aa6108, scope.project.id=f8933570c9dc44c8bb1ecbd74374889b, user.id=11ec991028e64f3db9ef62861c931821) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:29.408721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e61f3-867d-48c5-84a8-b1c2f460757b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:29.410607 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e61f3-867d-48c5-84a8-b1c2f460757b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:29.418909 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 87/179] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:48:29 2022] GET /identity/v3/role_assignments?user.id=11ec991028e64f3db9ef62861c931821&scope.project.id=f8933570c9dc44c8bb1ecbd74374889b&role.id=3631750e6bf34730aa3557cd90aa6108 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:29.456338 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74411fc-0352-4e99-9055-18396e70206c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:29.517198 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e509d-0754-401e-8a9f-dd64bfa7936e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:29.517882 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e509d-0754-401e-8a9f-dd64bfa7936e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:29.518242 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e509d-0754-401e-8a9f-dd64bfa7936e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:29.518524 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e509d-0754-401e-8a9f-dd64bfa7936e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:29.519394 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35e509d-0754-401e-8a9f-dd64bfa7936e None admin] RBAC: Adding query filter params (role.id=46ddc3cf4b124399a275bc639635e10a, scope.project.id=1a6770d749f644eea4fd1584b1503d28, user.id=11ec991028e64f3db9ef62861c931821) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:29.519701 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35e509d-0754-401e-8a9f-dd64bfa7936e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:29.521243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35e509d-0754-401e-8a9f-dd64bfa7936e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:29.531123 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 93/180] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:48:29 2022] GET /identity/v3/role_assignments?user.id=11ec991028e64f3db9ef62861c931821&scope.project.id=1a6770d749f644eea4fd1584b1503d28&role.id=46ddc3cf4b124399a275bc639635e10a => generated 608 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:29.673940 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db0333-5490-4f76-85e2-c249205849c1 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:29.674340 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db0333-5490-4f76-85e2-c249205849c1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:29.674743 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db0333-5490-4f76-85e2-c249205849c1 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:29.676616 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 88/181] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:29.683000 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4e7d8-b8c3-458b-a715-81e9617a4fd5 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:29.683501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4e7d8-b8c3-458b-a715-81e9617a4fd5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:29.683575 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4e7d8-b8c3-458b-a715-81e9617a4fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:29.702811 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d7a4e7d8-b8c3-458b-a715-81e9617a4fd5 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:29.704037 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d7a4e7d8-b8c3-458b-a715-81e9617a4fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:29.796389 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 94/182] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:29.820937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db0333-5490-4f76-85e2-c249205849c1 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:29.949070 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db0333-5490-4f76-85e2-c249205849c1 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:29.964188 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaebad7-4e52-450f-a277-a12e5b592e31 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:29.964188 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaebad7-4e52-450f-a277-a12e5b592e31 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:29.964188 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaebad7-4e52-450f-a277-a12e5b592e31 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:29.965414 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 95/183] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:48:29.973439 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec7a75-9635-4168-bdf0-83a57e9590b0 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:29.973724 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec7a75-9635-4168-bdf0-83a57e9590b0 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:29.974002 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec7a75-9635-4168-bdf0-83a57e9590b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:29.997344 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-67ec7a75-9635-4168-bdf0-83a57e9590b0 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:29.998544 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-67ec7a75-9635-4168-bdf0-83a57e9590b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:29.999827 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20debd5f-4e94-4413-bb05-9a1f2c22ac07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:30.000979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20debd5f-4e94-4413-bb05-9a1f2c22ac07 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:30.001483 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20debd5f-4e94-4413-bb05-9a1f2c22ac07 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:30.001956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20debd5f-4e94-4413-bb05-9a1f2c22ac07 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:30.015534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20debd5f-4e94-4413-bb05-9a1f2c22ac07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:30.026587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-20debd5f-4e94-4413-bb05-9a1f2c22ac07 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:48:30.026587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:30.026587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:30.026587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:30.026587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:30.026587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:30.026587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:30.026587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:30.026587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:30.026587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:30.026587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:30.026587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:30.026587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:30.026587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:30.026587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:30.026587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:30.027540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:30.027540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:30.027540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:30.027540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:30.027540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:30.027540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:30.027540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:30.027540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:30.027540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:30.027540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:30.027540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:30.027540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:48:30.027540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:30.027540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:30.027540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:30.027540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:30.028361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:30.028361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:30.028361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:30.028361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:30.028361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:30.028361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:30.028361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:30.028361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:30.028361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:30.028361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:30.028361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:48:30.028361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:48:30.030838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 89/184] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:48:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:30.039920 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20debd5f-4e94-4413-bb05-9a1f2c22ac07 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:30.081071 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 96/185] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:30.082512 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f891684-ca8c-49a9-b80c-b6a242c347c5 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:30.082796 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f891684-ca8c-49a9-b80c-b6a242c347c5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:30.083133 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f891684-ca8c-49a9-b80c-b6a242c347c5 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:30.084789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 97/186] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:30.091628 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469eeeaa-1077-475a-9141-777c99a93f04 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:30.091966 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469eeeaa-1077-475a-9141-777c99a93f04 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:30.092238 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469eeeaa-1077-475a-9141-777c99a93f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:30.108619 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-469eeeaa-1077-475a-9141-777c99a93f04 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:30.110062 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-469eeeaa-1077-475a-9141-777c99a93f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:30.159235 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20debd5f-4e94-4413-bb05-9a1f2c22ac07 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:30.186171 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 98/187] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:30.186171 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469eeeaa-1077-475a-9141-777c99a93f04 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:30.198294 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581203b-709c-44e7-9aca-6d6f9084aa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:30.199761 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581203b-709c-44e7-9aca-6d6f9084aa9a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:30.200294 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581203b-709c-44e7-9aca-6d6f9084aa9a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:30.200714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581203b-709c-44e7-9aca-6d6f9084aa9a None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:30.201774 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581203b-709c-44e7-9aca-6d6f9084aa9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:30.202620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581203b-709c-44e7-9aca-6d6f9084aa9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:30.204230 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581203b-709c-44e7-9aca-6d6f9084aa9a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:30.216530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 90/188] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:30.217705 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581203b-709c-44e7-9aca-6d6f9084aa9a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:30.290892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469eeeaa-1077-475a-9141-777c99a93f04 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:30.318664 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581203b-709c-44e7-9aca-6d6f9084aa9a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:30.323499 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa242a-ab42-489d-83dc-c9e6aded37fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:30.324461 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa242a-ab42-489d-83dc-c9e6aded37fb None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:30.324888 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa242a-ab42-489d-83dc-c9e6aded37fb None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:30.325305 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa242a-ab42-489d-83dc-c9e6aded37fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:30.326673 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffa242a-ab42-489d-83dc-c9e6aded37fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:30.328288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffa242a-ab42-489d-83dc-c9e6aded37fb None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:30.330383 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 99/189] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:48:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:30.331288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa242a-ab42-489d-83dc-c9e6aded37fb None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:30.356633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8c0d6-6ff3-4039-a1b4-c520098050d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:30.357436 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8c0d6-6ff3-4039-a1b4-c520098050d6 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:30.357715 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8c0d6-6ff3-4039-a1b4-c520098050d6 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:30.357985 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8c0d6-6ff3-4039-a1b4-c520098050d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:30.368099 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf8c0d6-6ff3-4039-a1b4-c520098050d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:30.376105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-5cf8c0d6-6ff3-4039-a1b4-c520098050d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:48:30.376105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:30.376105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:30.376105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:30.376105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:30.376105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:30.376105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:30.376105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:30.376105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:30.376105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:30.376105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:30.376105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:30.376105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:30.376105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:30.376105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:30.376105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:30.377133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:30.377133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:30.377133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:30.377133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:30.377133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:30.377133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:30.377133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:30.377133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:30.377133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:30.377133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:30.377133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:30.377133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:48:30.377133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:30.377133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:30.377133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:30.377133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:30.377959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:30.377959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:30.377959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:30.377959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:30.377959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:30.377959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:30.377959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:30.377959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:30.377959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:30.377959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:30.377959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:48:30.377959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:48:30.377959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 91/190] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 9 22:48:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:30.378641 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8c0d6-6ff3-4039-a1b4-c520098050d6 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:30.433068 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa242a-ab42-489d-83dc-c9e6aded37fb None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:30.467566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56aad2-1b29-4667-9b5e-a7a0e627fc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:30.468655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56aad2-1b29-4667-9b5e-a7a0e627fc9f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:30.469079 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56aad2-1b29-4667-9b5e-a7a0e627fc9f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:30.469507 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56aad2-1b29-4667-9b5e-a7a0e627fc9f None admin] PATH_INFO: `/v3/users/31c77c49b318466f8c82caa90facbf35` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:30.479230 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8c0d6-6ff3-4039-a1b4-c520098050d6 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:30.485360 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa56aad2-1b29-4667-9b5e-a7a0e627fc9f None admin] RBAC: Authorizing `identity:get_user(user_id=31c77c49b318466f8c82caa90facbf35)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:30.487437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa56aad2-1b29-4667-9b5e-a7a0e627fc9f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:30.489988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 100/191] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:30 2022] GET /identity/v3/users/31c77c49b318466f8c82caa90facbf35 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:30.490905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56aad2-1b29-4667-9b5e-a7a0e627fc9f None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:30.516420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24544b-ebed-4099-a73a-75e8ba1a778e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:30.517110 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24544b-ebed-4099-a73a-75e8ba1a778e None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:30.517392 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24544b-ebed-4099-a73a-75e8ba1a778e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:30.517672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24544b-ebed-4099-a73a-75e8ba1a778e None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:30.518398 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d24544b-ebed-4099-a73a-75e8ba1a778e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:30.519858 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d24544b-ebed-4099-a73a-75e8ba1a778e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:30.546791 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 92/192] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 9 22:48:30 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:48:30.548213 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24544b-ebed-4099-a73a-75e8ba1a778e None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:30.617512 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56aad2-1b29-4667-9b5e-a7a0e627fc9f None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:30.662810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0310631-ae16-4046-9776-602aac3b3483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:30.664312 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0310631-ae16-4046-9776-602aac3b3483 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:30.664800 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0310631-ae16-4046-9776-602aac3b3483 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:30.665235 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0310631-ae16-4046-9776-602aac3b3483 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:30.665705 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24544b-ebed-4099-a73a-75e8ba1a778e None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:30.666367 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0310631-ae16-4046-9776-602aac3b3483 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:30.666815 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0310631-ae16-4046-9776-602aac3b3483 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:30.669528 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0310631-ae16-4046-9776-602aac3b3483 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:30.685594 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 101/193] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:48:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:30.688219 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0310631-ae16-4046-9776-602aac3b3483 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:30.713756 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962d5dc-2500-40d8-8f86-1918f8963a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:30.714886 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962d5dc-2500-40d8-8f86-1918f8963a18 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:30.715267 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962d5dc-2500-40d8-8f86-1918f8963a18 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:30.715646 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962d5dc-2500-40d8-8f86-1918f8963a18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:30.793700 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0962d5dc-2500-40d8-8f86-1918f8963a18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:30.796126 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0962d5dc-2500-40d8-8f86-1918f8963a18 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:30.809647 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0310631-ae16-4046-9776-602aac3b3483 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:30.856097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d233a81-5617-484e-911d-c2e7e3a8b19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:30.856097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d233a81-5617-484e-911d-c2e7e3a8b19e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:30.856097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d233a81-5617-484e-911d-c2e7e3a8b19e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:30.856097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d233a81-5617-484e-911d-c2e7e3a8b19e None admin] PATH_INFO: `/v3/users/31c77c49b318466f8c82caa90facbf35` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:30.857234 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d233a81-5617-484e-911d-c2e7e3a8b19e None admin] RBAC: Authorizing `identity:get_user(user_id=31c77c49b318466f8c82caa90facbf35)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:30.858444 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d233a81-5617-484e-911d-c2e7e3a8b19e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:30.860351 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 102/194] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:30 2022] GET /identity/v3/users/31c77c49b318466f8c82caa90facbf35 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:30.865921 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d233a81-5617-484e-911d-c2e7e3a8b19e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:30.928053 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 93/195] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:48:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:48:30.934749 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962d5dc-2500-40d8-8f86-1918f8963a18 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:31.005101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d233a81-5617-484e-911d-c2e7e3a8b19e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:31.081033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962d5dc-2500-40d8-8f86-1918f8963a18 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:31.087367 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e013a1c-5a59-47cc-bf71-0e0665a55558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:31.088244 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e013a1c-5a59-47cc-bf71-0e0665a55558 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:31.088702 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e013a1c-5a59-47cc-bf71-0e0665a55558 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:31.089187 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e013a1c-5a59-47cc-bf71-0e0665a55558 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:31.118355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d87826-45e0-4119-835b-844606a485e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:31.119233 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d87826-45e0-4119-835b-844606a485e5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:31.119537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d87826-45e0-4119-835b-844606a485e5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:31.119815 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d87826-45e0-4119-835b-844606a485e5 None admin] PATH_INFO: `/v3/projects/db7334eab91d41fa884c491d1364a253` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:31.120997 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d87826-45e0-4119-835b-844606a485e5 None admin] RBAC: Authorizing `identity:get_project(project_id=db7334eab91d41fa884c491d1364a253)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:31.122469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d87826-45e0-4119-835b-844606a485e5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:31.128952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 94/196] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:30 2022] GET /identity/v3/projects/db7334eab91d41fa884c491d1364a253 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:31.130139 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d87826-45e0-4119-835b-844606a485e5 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:31.235828 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e013a1c-5a59-47cc-bf71-0e0665a55558 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:31.237626 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e013a1c-5a59-47cc-bf71-0e0665a55558 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:31.265950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d87826-45e0-4119-835b-844606a485e5 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:31.304404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61b2a4-d119-4aac-b90d-989a099cbe62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:31.305357 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61b2a4-d119-4aac-b90d-989a099cbe62 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:31.307742 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61b2a4-d119-4aac-b90d-989a099cbe62 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:31.308207 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61b2a4-d119-4aac-b90d-989a099cbe62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:31.309225 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e61b2a4-d119-4aac-b90d-989a099cbe62 None admin] RBAC: Adding query filter params (role.id=bda550492f4f4c5c83f52fa89fc18a7d, scope.project.id=db7334eab91d41fa884c491d1364a253, user.id=31c77c49b318466f8c82caa90facbf35) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:31.309718 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e61b2a4-d119-4aac-b90d-989a099cbe62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:31.311382 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e61b2a4-d119-4aac-b90d-989a099cbe62 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:31.327657 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 95/197] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:48:31 2022] GET /identity/v3/role_assignments?user.id=31c77c49b318466f8c82caa90facbf35&scope.project.id=db7334eab91d41fa884c491d1364a253&role.id=bda550492f4f4c5c83f52fa89fc18a7d => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:31.391573 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 103/198] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:48:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 22:48:31.399340 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61b2a4-d119-4aac-b90d-989a099cbe62 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:31.543799 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61b2a4-d119-4aac-b90d-989a099cbe62 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:31.580237 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b2f72-52a2-4072-89f6-982221776e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:31.581473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b2f72-52a2-4072-89f6-982221776e01 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:31.581988 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b2f72-52a2-4072-89f6-982221776e01 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:31.582497 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b2f72-52a2-4072-89f6-982221776e01 None admin] PATH_INFO: `/v3/projects/db7334eab91d41fa884c491d1364a253` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:31.583893 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b2f72-52a2-4072-89f6-982221776e01 None admin] RBAC: Authorizing `identity:get_project(project_id=db7334eab91d41fa884c491d1364a253)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:31.585637 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b2f72-52a2-4072-89f6-982221776e01 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:31.587634 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 96/199] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:31 2022] GET /identity/v3/projects/db7334eab91d41fa884c491d1364a253 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:31.593490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e013a1c-5a59-47cc-bf71-0e0665a55558 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:31.703049 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e013a1c-5a59-47cc-bf71-0e0665a55558 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:31.738827 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f287370-12f6-4d75-b2f9-24896b451152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:31.739482 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f287370-12f6-4d75-b2f9-24896b451152 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:31.739764 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f287370-12f6-4d75-b2f9-24896b451152 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:31.740037 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f287370-12f6-4d75-b2f9-24896b451152 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:31.740858 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f287370-12f6-4d75-b2f9-24896b451152 None admin] RBAC: Adding query filter params (role.id=46ddc3cf4b124399a275bc639635e10a, scope.project.id=db7334eab91d41fa884c491d1364a253, user.id=31c77c49b318466f8c82caa90facbf35) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:31.741169 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f287370-12f6-4d75-b2f9-24896b451152 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:31.742711 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f287370-12f6-4d75-b2f9-24896b451152 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:31.748608 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 104/200] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:48:31 2022] GET /identity/v3/role_assignments?user.id=31c77c49b318466f8c82caa90facbf35&scope.project.id=db7334eab91d41fa884c491d1364a253&role.id=46ddc3cf4b124399a275bc639635e10a => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:32.097152 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abaedc-b8fc-44ac-ba4d-f088f3527958 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:32.098219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abaedc-b8fc-44ac-ba4d-f088f3527958 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:32.098744 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abaedc-b8fc-44ac-ba4d-f088f3527958 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:32.100422 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 97/201] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:32.106461 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40dbaf5-9480-493d-b4db-346d08c5c866 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:32.107047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40dbaf5-9480-493d-b4db-346d08c5c866 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:32.107516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40dbaf5-9480-493d-b4db-346d08c5c866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:32.125460 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b40dbaf5-9480-493d-b4db-346d08c5c866 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:32.126938 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b40dbaf5-9480-493d-b4db-346d08c5c866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:32.211931 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 105/202] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:32.245355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abaedc-b8fc-44ac-ba4d-f088f3527958 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:32.367476 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abaedc-b8fc-44ac-ba4d-f088f3527958 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:32.409450 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c6fad-26cd-43c8-ab2b-a289139de80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:32.410273 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c6fad-26cd-43c8-ab2b-a289139de80d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:32.410549 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c6fad-26cd-43c8-ab2b-a289139de80d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:32.410816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c6fad-26cd-43c8-ab2b-a289139de80d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:32.426797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c6fad-26cd-43c8-ab2b-a289139de80d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:32.434381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-351c6fad-26cd-43c8-ab2b-a289139de80d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 22:48:32.434381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:32.434381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:32.434381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:32.434381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:32.434381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:32.434381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:32.434381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:32.434381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:32.434381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:32.434381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:32.434381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:32.434381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:32.434381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:32.434381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:32.434381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:32.435257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:32.435257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:32.435257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:32.435257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:32.435257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:32.435257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:32.435257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:32.435257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:32.435257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:32.435257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:32.435257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:32.435257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:48:32.435257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:32.435257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:32.435257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:32.435257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:32.436055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:32.436055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:32.436055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:32.436055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:32.436055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:32.436055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:32.436055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:32.436055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:32.436055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:32.436055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:32.436055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 22:48:32.436055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:48:32.436055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 98/203] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:32.442833 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40dbaf5-9480-493d-b4db-346d08c5c866 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:32.613720 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40dbaf5-9480-493d-b4db-346d08c5c866 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:32.665630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70321265-2f2e-4d7b-8f10-041084ac6c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:32.666695 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70321265-2f2e-4d7b-8f10-041084ac6c3f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:32.667174 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70321265-2f2e-4d7b-8f10-041084ac6c3f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:32.667675 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70321265-2f2e-4d7b-8f10-041084ac6c3f None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:32.668666 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70321265-2f2e-4d7b-8f10-041084ac6c3f None admin] RBAC: Adding query filter params (name=reader) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:32.669188 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70321265-2f2e-4d7b-8f10-041084ac6c3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:32.670776 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70321265-2f2e-4d7b-8f10-041084ac6c3f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:32.680315 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 106/204] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:32.687184 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c6fad-26cd-43c8-ab2b-a289139de80d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:32.833435 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c6fad-26cd-43c8-ab2b-a289139de80d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:32.885850 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58606349-70d7-427c-b59f-9a31e216b450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:32.886673 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58606349-70d7-427c-b59f-9a31e216b450 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:32.886953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58606349-70d7-427c-b59f-9a31e216b450 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:32.887226 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58606349-70d7-427c-b59f-9a31e216b450 None admin] PATH_INFO: `/v3/users/e627cb592eca48b482cb248bd3f1f207` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:32.890819 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58606349-70d7-427c-b59f-9a31e216b450 None admin] RBAC: Authorizing `identity:get_user(user_id=e627cb592eca48b482cb248bd3f1f207)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:32.892276 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58606349-70d7-427c-b59f-9a31e216b450 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:32.894093 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 99/205] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:32 2022] GET /identity/v3/users/e627cb592eca48b482cb248bd3f1f207 => generated 289 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:32.899757 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70321265-2f2e-4d7b-8f10-041084ac6c3f None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:33.038153 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70321265-2f2e-4d7b-8f10-041084ac6c3f None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:33.077481 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92f33a-3981-4563-bc23-98b574cf4b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:33.078526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92f33a-3981-4563-bc23-98b574cf4b42 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:33.079007 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92f33a-3981-4563-bc23-98b574cf4b42 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:33.079471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92f33a-3981-4563-bc23-98b574cf4b42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:33.171581 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d92f33a-3981-4563-bc23-98b574cf4b42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:33.173369 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d92f33a-3981-4563-bc23-98b574cf4b42 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:33.275657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 107/206] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:48:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:48:33.283243 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58606349-70d7-427c-b59f-9a31e216b450 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:33.438523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58606349-70d7-427c-b59f-9a31e216b450 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:33.441975 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f27939-4484-457d-9a51-1132ca823039 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:33.442300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f27939-4484-457d-9a51-1132ca823039 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:33.442580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f27939-4484-457d-9a51-1132ca823039 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:33.444214 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 108/207] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:33.477197 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06514d-374e-471a-bf57-368f9dac7ed2 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:33.477491 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06514d-374e-471a-bf57-368f9dac7ed2 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:33.477748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06514d-374e-471a-bf57-368f9dac7ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:33.479001 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99db9a1-00e6-472f-bfe1-b20404f1d82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:33.479858 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99db9a1-00e6-472f-bfe1-b20404f1d82a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:33.480142 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99db9a1-00e6-472f-bfe1-b20404f1d82a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:33.480422 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99db9a1-00e6-472f-bfe1-b20404f1d82a None admin] PATH_INFO: `/v3/projects/1a6770d749f644eea4fd1584b1503d28` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:33.481601 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99db9a1-00e6-472f-bfe1-b20404f1d82a None admin] RBAC: Authorizing `identity:get_project(project_id=1a6770d749f644eea4fd1584b1503d28)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:33.483091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99db9a1-00e6-472f-bfe1-b20404f1d82a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:33.485443 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 100/208] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:33 2022] GET /identity/v3/projects/1a6770d749f644eea4fd1584b1503d28 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:33.495071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99db9a1-00e6-472f-bfe1-b20404f1d82a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:33.505741 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-5e06514d-374e-471a-bf57-368f9dac7ed2 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:33.506985 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5e06514d-374e-471a-bf57-368f9dac7ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:33.588314 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 109/209] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:33.598874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3de31-ed7a-45a8-bd96-4aaf88e112c2 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:33.599160 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3de31-ed7a-45a8-bd96-4aaf88e112c2 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:33.599439 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3de31-ed7a-45a8-bd96-4aaf88e112c2 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:33.601559 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 110/210] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:48:33.607247 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9552aa-36de-44ed-8466-4ffade618a19 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:33.607762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9552aa-36de-44ed-8466-4ffade618a19 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:33.608230 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9552aa-36de-44ed-8466-4ffade618a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:33.633202 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ac9552aa-36de-44ed-8466-4ffade618a19 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:33.636393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ac9552aa-36de-44ed-8466-4ffade618a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:33.643065 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99db9a1-00e6-472f-bfe1-b20404f1d82a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:33.683157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed819f-8bcc-492e-a0c4-d7792629947e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:33.684590 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed819f-8bcc-492e-a0c4-d7792629947e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:33.685153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed819f-8bcc-492e-a0c4-d7792629947e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:33.685636 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed819f-8bcc-492e-a0c4-d7792629947e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:33.687051 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed819f-8bcc-492e-a0c4-d7792629947e None admin] RBAC: Adding query filter params (role.id=949d4917fcdb41e2a938aadb277d1970, scope.project.id=1a6770d749f644eea4fd1584b1503d28, user.id=e627cb592eca48b482cb248bd3f1f207) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:33.687897 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed819f-8bcc-492e-a0c4-d7792629947e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:33.690049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed819f-8bcc-492e-a0c4-d7792629947e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:33.698538 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 101/211] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:48:33 2022] GET /identity/v3/role_assignments?user.id=e627cb592eca48b482cb248bd3f1f207&scope.project.id=1a6770d749f644eea4fd1584b1503d28&role.id=949d4917fcdb41e2a938aadb277d1970 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:33.706149 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed819f-8bcc-492e-a0c4-d7792629947e None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:33.713673 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 111/212] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:48:33.743503 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9552aa-36de-44ed-8466-4ffade618a19 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:33.863137 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9552aa-36de-44ed-8466-4ffade618a19 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:33.900408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed819f-8bcc-492e-a0c4-d7792629947e None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:33.903833 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1a49b-da33-4eb5-a4c9-f37c94ba0858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:33.904642 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1a49b-da33-4eb5-a4c9-f37c94ba0858 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:33.904923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1a49b-da33-4eb5-a4c9-f37c94ba0858 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:33.905197 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1a49b-da33-4eb5-a4c9-f37c94ba0858 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:33.907335 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1a49b-da33-4eb5-a4c9-f37c94ba0858 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:33.908810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1a49b-da33-4eb5-a4c9-f37c94ba0858 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:33.910821 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 112/213] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:48:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:33.916857 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1a49b-da33-4eb5-a4c9-f37c94ba0858 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:33.968160 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7d16e-13d1-4619-9195-1c980be7a875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:33.968888 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7d16e-13d1-4619-9195-1c980be7a875 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:33.969167 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7d16e-13d1-4619-9195-1c980be7a875 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:33.969438 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7d16e-13d1-4619-9195-1c980be7a875 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:33.995166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7d16e-13d1-4619-9195-1c980be7a875 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:34.009367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-9af7d16e-13d1-4619-9195-1c980be7a875 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:34.009367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:34.009367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:34.009367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:34.009367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:34.009367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:34.009367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:34.009367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:34.009367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:34.009367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:34.009367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:34.009367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:34.009367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:34.009367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:34.009367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:34.009367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:34.010364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:34.010364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:34.010364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:34.010364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:34.010364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:34.010364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:34.010364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:34.010364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:34.010364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:34.010364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:34.010364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:34.010364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:48:34.010364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:34.010364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:34.010364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:34.010364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:34.011175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:34.011175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:34.011175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:34.011175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:34.011175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:34.011175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:34.011175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:34.011175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:34.011175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:34.011175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:34.011175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:34.011175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:48:34.011175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 102/214] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:34.018004 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7d16e-13d1-4619-9195-1c980be7a875 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:34.029632 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1a49b-da33-4eb5-a4c9-f37c94ba0858 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:34.067016 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834fbbbd-d0bd-44bb-bf07-ad51fc90d8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:34.068531 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fbbbd-d0bd-44bb-bf07-ad51fc90d8fb None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:34.068531 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fbbbd-d0bd-44bb-bf07-ad51fc90d8fb None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:34.068531 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fbbbd-d0bd-44bb-bf07-ad51fc90d8fb None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:34.068975 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834fbbbd-d0bd-44bb-bf07-ad51fc90d8fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:34.072067 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834fbbbd-d0bd-44bb-bf07-ad51fc90d8fb None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:34.103847 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 113/215] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 9 22:48:33 2022] POST /identity/v3/users => generated 297 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:48:34.158324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7d16e-13d1-4619-9195-1c980be7a875 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:34.198593 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af91b1-c63d-43f2-b21e-ef55f7a3ed9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:34.199644 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af91b1-c63d-43f2-b21e-ef55f7a3ed9e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:34.200127 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af91b1-c63d-43f2-b21e-ef55f7a3ed9e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:34.200652 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af91b1-c63d-43f2-b21e-ef55f7a3ed9e None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:34.201708 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0af91b1-c63d-43f2-b21e-ef55f7a3ed9e None admin] RBAC: Adding query filter params (name=member) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:34.202240 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0af91b1-c63d-43f2-b21e-ef55f7a3ed9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:34.203934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0af91b1-c63d-43f2-b21e-ef55f7a3ed9e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:34.217260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 103/216] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:34.220423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834fbbbd-d0bd-44bb-bf07-ad51fc90d8fb None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:34.283261 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e2006-a91f-40ee-9d65-35eb33e0dd77 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:34.283798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e2006-a91f-40ee-9d65-35eb33e0dd77 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:34.283798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e2006-a91f-40ee-9d65-35eb33e0dd77 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:34.285434 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 104/217] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:48:34.293005 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f2c84-84c1-4df6-bc78-2f738a565cfc None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:34.293005 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f2c84-84c1-4df6-bc78-2f738a565cfc None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:34.293005 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f2c84-84c1-4df6-bc78-2f738a565cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:34.314419 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-365f2c84-84c1-4df6-bc78-2f738a565cfc None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:34.316305 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-365f2c84-84c1-4df6-bc78-2f738a565cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:34.348336 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834fbbbd-d0bd-44bb-bf07-ad51fc90d8fb None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:34.390539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031573b-f234-4eb7-9fb3-9dc8510e46fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:34.392109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031573b-f234-4eb7-9fb3-9dc8510e46fb None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:34.392388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031573b-f234-4eb7-9fb3-9dc8510e46fb None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:34.392662 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031573b-f234-4eb7-9fb3-9dc8510e46fb None admin] PATH_INFO: `/v3/users/08f98fc5803d4101a40d48ae3945a058` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:34.405018 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 105/218] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:34.412703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031573b-f234-4eb7-9fb3-9dc8510e46fb None admin] RBAC: Authorizing `identity:get_user(user_id=08f98fc5803d4101a40d48ae3945a058)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:34.413152 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031573b-f234-4eb7-9fb3-9dc8510e46fb None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:34.415104 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 114/219] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:34 2022] GET /identity/v3/users/08f98fc5803d4101a40d48ae3945a058 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:34.421268 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365f2c84-84c1-4df6-bc78-2f738a565cfc None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:34.427864 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031573b-f234-4eb7-9fb3-9dc8510e46fb None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:34.544084 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365f2c84-84c1-4df6-bc78-2f738a565cfc None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:34.557892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031573b-f234-4eb7-9fb3-9dc8510e46fb None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:34.590428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d84a0-7655-400f-b19b-aae6259b07a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:34.591213 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d84a0-7655-400f-b19b-aae6259b07a4 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:34.591495 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d84a0-7655-400f-b19b-aae6259b07a4 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:34.591764 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d84a0-7655-400f-b19b-aae6259b07a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:34.597960 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130302ad-835b-492b-9b9c-c9763b6a0a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:34.599025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130302ad-835b-492b-9b9c-c9763b6a0a42 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:34.599503 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130302ad-835b-492b-9b9c-c9763b6a0a42 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:34.599967 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130302ad-835b-492b-9b9c-c9763b6a0a42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:34.614019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130302ad-835b-492b-9b9c-c9763b6a0a42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:34.624541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-130302ad-835b-492b-9b9c-c9763b6a0a42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:48:34.624541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:34.624541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:34.624541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:34.624541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:34.624541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:34.624541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:34.624541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:34.624541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:34.624541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:34.624541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:34.624541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:34.624541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:34.624541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:34.624541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:34.624541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:34.625443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:34.625443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:34.625443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:34.625443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:34.625443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:34.625443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:34.625443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:34.625443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:34.625443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:34.625443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:34.625443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:34.625443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:34.625443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:34.625443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:34.625443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:34.625443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:34.626278 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:34.626278 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:34.626278 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:34.626278 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:34.626278 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:34.626278 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:34.626278 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:34.626278 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:34.626278 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:34.626278 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:34.626278 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:48:34.626278 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:34.628228 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 115/220] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:48:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:34.630939 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0373441-dcac-461d-90d5-42592ef52b50 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:34.632815 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0373441-dcac-461d-90d5-42592ef52b50 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:34.633518 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0373441-dcac-461d-90d5-42592ef52b50 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:34.635345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 116/221] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:34.636274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0373441-dcac-461d-90d5-42592ef52b50 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:34.679789 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d84a0-7655-400f-b19b-aae6259b07a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:34.681283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d84a0-7655-400f-b19b-aae6259b07a4 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:34.749379 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0373441-dcac-461d-90d5-42592ef52b50 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:34.791700 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25af44-befa-4d54-8fa9-71ae5be8e56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:34.792366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25af44-befa-4d54-8fa9-71ae5be8e56c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:34.792642 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25af44-befa-4d54-8fa9-71ae5be8e56c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:34.792914 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25af44-befa-4d54-8fa9-71ae5be8e56c None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:34.793653 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c25af44-befa-4d54-8fa9-71ae5be8e56c None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:34.793951 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c25af44-befa-4d54-8fa9-71ae5be8e56c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:34.795526 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 106/222] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:48:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:48:34.795635 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c25af44-befa-4d54-8fa9-71ae5be8e56c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:34.797251 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d4ab8-0310-4ed7-bc7b-bfc7ef448c5d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:34.797783 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d4ab8-0310-4ed7-bc7b-bfc7ef448c5d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:34.798309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d4ab8-0310-4ed7-bc7b-bfc7ef448c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:34.806862 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 117/223] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:34.807719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25af44-befa-4d54-8fa9-71ae5be8e56c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:34.816752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f27d4ab8-0310-4ed7-bc7b-bfc7ef448c5d None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:34.818422 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f27d4ab8-0310-4ed7-bc7b-bfc7ef448c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:34.889523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 107/224] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:34.890784 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d4ab8-0310-4ed7-bc7b-bfc7ef448c5d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:34.941756 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25af44-befa-4d54-8fa9-71ae5be8e56c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:34.986216 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec017d-3f7c-45be-98bd-b590ff9ff40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:34.987098 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec017d-3f7c-45be-98bd-b590ff9ff40f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:34.987602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec017d-3f7c-45be-98bd-b590ff9ff40f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:34.987877 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec017d-3f7c-45be-98bd-b590ff9ff40f None admin] PATH_INFO: `/v3/projects/db7334eab91d41fa884c491d1364a253` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:34.989199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec017d-3f7c-45be-98bd-b590ff9ff40f None admin] RBAC: Authorizing `identity:get_project(project_id=db7334eab91d41fa884c491d1364a253)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:34.991043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec017d-3f7c-45be-98bd-b590ff9ff40f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:34.992827 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 118/225] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:34 2022] GET /identity/v3/projects/db7334eab91d41fa884c491d1364a253 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:34.994285 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec017d-3f7c-45be-98bd-b590ff9ff40f None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:35.005454 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d4ab8-0310-4ed7-bc7b-bfc7ef448c5d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:35.040505 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1edee4-d926-4ec7-8391-35681943b756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:35.041863 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1edee4-d926-4ec7-8391-35681943b756 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:35.042380 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1edee4-d926-4ec7-8391-35681943b756 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:35.042852 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1edee4-d926-4ec7-8391-35681943b756 None admin] PATH_INFO: `/v3/users/31c77c49b318466f8c82caa90facbf35` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:35.044301 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1edee4-d926-4ec7-8391-35681943b756 None admin] RBAC: Authorizing `identity:get_user(user_id=31c77c49b318466f8c82caa90facbf35)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:35.046150 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1edee4-d926-4ec7-8391-35681943b756 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:35.048122 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 108/226] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:34 2022] GET /identity/v3/users/31c77c49b318466f8c82caa90facbf35 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:35.049080 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1edee4-d926-4ec7-8391-35681943b756 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:35.113342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec017d-3f7c-45be-98bd-b590ff9ff40f None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:35.157433 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d2137-ffb9-4dec-8b1f-9cae56d61b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:35.158658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d2137-ffb9-4dec-8b1f-9cae56d61b73 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:35.159034 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d2137-ffb9-4dec-8b1f-9cae56d61b73 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:35.159358 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d2137-ffb9-4dec-8b1f-9cae56d61b73 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:35.161725 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1edee4-d926-4ec7-8391-35681943b756 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:35.172618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d2137-ffb9-4dec-8b1f-9cae56d61b73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:35.183091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-f97d2137-ffb9-4dec-8b1f-9cae56d61b73 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:48:35.183091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:35.183091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:35.183091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:35.183091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:35.183091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:35.183091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:35.183091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:35.183091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:35.183091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:35.183091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:35.183091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:35.183091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:35.183091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:35.183091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:35.183091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:35.183998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:35.183998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:35.183998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:35.183998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:35.183998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:35.183998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:35.183998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:35.183998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:35.183998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:35.183998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:35.183998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:35.183998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:35.183998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:35.183998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:35.183998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:35.183998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:35.184791 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:35.184791 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:35.184791 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:35.184791 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:35.184791 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:35.184791 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:35.184791 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:35.184791 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:35.184791 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:35.184791 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:35.184791 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:48:35.184791 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:35.189166 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 119/227] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 9 22:48:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:35.190228 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d2137-ffb9-4dec-8b1f-9cae56d61b73 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:35.202859 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33405b12-b891-4c26-9476-1c130bf4122e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:35.203898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33405b12-b891-4c26-9476-1c130bf4122e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:35.204380 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33405b12-b891-4c26-9476-1c130bf4122e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:35.204882 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33405b12-b891-4c26-9476-1c130bf4122e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:35.205958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33405b12-b891-4c26-9476-1c130bf4122e None admin] RBAC: Adding query filter params (role.id=3631750e6bf34730aa3557cd90aa6108, scope.project.id=db7334eab91d41fa884c491d1364a253, user.id=08f98fc5803d4101a40d48ae3945a058) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:35.206551 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33405b12-b891-4c26-9476-1c130bf4122e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:35.208350 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33405b12-b891-4c26-9476-1c130bf4122e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:35.215364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 109/228] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:48:35 2022] GET /identity/v3/role_assignments?user.id=08f98fc5803d4101a40d48ae3945a058&scope.project.id=db7334eab91d41fa884c491d1364a253&role.id=3631750e6bf34730aa3557cd90aa6108 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:35.216504 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33405b12-b891-4c26-9476-1c130bf4122e None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:35.312221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d2137-ffb9-4dec-8b1f-9cae56d61b73 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:35.345984 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33405b12-b891-4c26-9476-1c130bf4122e None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:35.359948 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a6cba-dfee-4851-adc0-bc0193e81763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:35.359948 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a6cba-dfee-4851-adc0-bc0193e81763 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:35.360254 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a6cba-dfee-4851-adc0-bc0193e81763 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:35.360488 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a6cba-dfee-4851-adc0-bc0193e81763 None admin] PATH_INFO: `/v3/projects/db7334eab91d41fa884c491d1364a253` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:35.361725 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829a6cba-dfee-4851-adc0-bc0193e81763 None admin] RBAC: Authorizing `identity:get_project(project_id=db7334eab91d41fa884c491d1364a253)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:35.363277 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829a6cba-dfee-4851-adc0-bc0193e81763 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:35.365035 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 120/229] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:35 2022] GET /identity/v3/projects/db7334eab91d41fa884c491d1364a253 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:35.370974 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a6cba-dfee-4851-adc0-bc0193e81763 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:35.389319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c28545-bddb-411a-885b-917a90ef3c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:35.390428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c28545-bddb-411a-885b-917a90ef3c57 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:35.390910 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c28545-bddb-411a-885b-917a90ef3c57 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:35.391378 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c28545-bddb-411a-885b-917a90ef3c57 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:35.392319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c28545-bddb-411a-885b-917a90ef3c57 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:35.392830 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c28545-bddb-411a-885b-917a90ef3c57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:35.394445 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c28545-bddb-411a-885b-917a90ef3c57 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:35.403650 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 110/230] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:48:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:35.415420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c28545-bddb-411a-885b-917a90ef3c57 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:35.478262 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a6cba-dfee-4851-adc0-bc0193e81763 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:35.513279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c6e8d-b06e-46ec-a0f1-02dd741ac588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:35.514308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c6e8d-b06e-46ec-a0f1-02dd741ac588 None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:35.514594 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c6e8d-b06e-46ec-a0f1-02dd741ac588 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:35.514867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c6e8d-b06e-46ec-a0f1-02dd741ac588 None admin] PATH_INFO: `/v3/projects/db7334eab91d41fa884c491d1364a253/users/31c77c49b318466f8c82caa90facbf35/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:35.516867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c6e8d-b06e-46ec-a0f1-02dd741ac588 None admin] RBAC: Authorizing `identity:create_grant(project_id=db7334eab91d41fa884c491d1364a253, user_id=31c77c49b318466f8c82caa90facbf35, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:35.518344 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c6e8d-b06e-46ec-a0f1-02dd741ac588 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:35.535328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 121/231] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:48:35 2022] PUT /identity/v3/projects/db7334eab91d41fa884c491d1364a253/users/31c77c49b318466f8c82caa90facbf35/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 22:48:35.546244 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c28545-bddb-411a-885b-917a90ef3c57 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:35.604319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e5692-987d-473e-9c75-8baab8dcb746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:35.605600 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e5692-987d-473e-9c75-8baab8dcb746 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:35.606081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e5692-987d-473e-9c75-8baab8dcb746 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:35.606642 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e5692-987d-473e-9c75-8baab8dcb746 None admin] PATH_INFO: `/v3/users/31c77c49b318466f8c82caa90facbf35` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:35.608387 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e5692-987d-473e-9c75-8baab8dcb746 None admin] RBAC: Authorizing `identity:get_user(user_id=31c77c49b318466f8c82caa90facbf35)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:35.610453 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e5692-987d-473e-9c75-8baab8dcb746 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:35.612606 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 111/232] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:35 2022] GET /identity/v3/users/31c77c49b318466f8c82caa90facbf35 => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:35.618701 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c6e8d-b06e-46ec-a0f1-02dd741ac588 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:35.760236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c6e8d-b06e-46ec-a0f1-02dd741ac588 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:35.795143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9de560-40b3-4a26-a1d9-03967bc0ccad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:35.796007 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9de560-40b3-4a26-a1d9-03967bc0ccad None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:35.796286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9de560-40b3-4a26-a1d9-03967bc0ccad None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:35.796555 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9de560-40b3-4a26-a1d9-03967bc0ccad None admin] PATH_INFO: `/v3/projects/db7334eab91d41fa884c491d1364a253` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:35.797904 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9de560-40b3-4a26-a1d9-03967bc0ccad None admin] RBAC: Authorizing `identity:get_project(project_id=db7334eab91d41fa884c491d1364a253)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:35.799416 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9de560-40b3-4a26-a1d9-03967bc0ccad None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:35.801485 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 122/233] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:35 2022] GET /identity/v3/projects/db7334eab91d41fa884c491d1364a253 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:35.807011 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e5692-987d-473e-9c75-8baab8dcb746 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:35.914825 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e5692-987d-473e-9c75-8baab8dcb746 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:35.952714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a84b9-8dc5-420e-aa1b-ab5f0c9850ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:35.953660 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a84b9-8dc5-420e-aa1b-ab5f0c9850ff None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:35.953944 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a84b9-8dc5-420e-aa1b-ab5f0c9850ff None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:35.954246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a84b9-8dc5-420e-aa1b-ab5f0c9850ff None admin] PATH_INFO: `/v3/projects/db7334eab91d41fa884c491d1364a253/users/31c77c49b318466f8c82caa90facbf35/roles/46ddc3cf4b124399a275bc639635e10a` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:35.956535 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a84b9-8dc5-420e-aa1b-ab5f0c9850ff None admin] RBAC: Authorizing `identity:create_grant(project_id=db7334eab91d41fa884c491d1364a253, user_id=31c77c49b318466f8c82caa90facbf35, role_id=46ddc3cf4b124399a275bc639635e10a)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:35.957996 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a84b9-8dc5-420e-aa1b-ab5f0c9850ff None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:35.989794 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 112/234] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:48:35 2022] PUT /identity/v3/projects/db7334eab91d41fa884c491d1364a253/users/31c77c49b318466f8c82caa90facbf35/roles/46ddc3cf4b124399a275bc639635e10a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 22:48:37.002425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745cff9-da64-4337-95a2-890c608719e8 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:37.003425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745cff9-da64-4337-95a2-890c608719e8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:37.003912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745cff9-da64-4337-95a2-890c608719e8 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:37.005726 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 123/235] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:37.012250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd1655-cf5a-4b15-958a-8baac140243f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:37.013071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd1655-cf5a-4b15-958a-8baac140243f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:37.013571 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd1655-cf5a-4b15-958a-8baac140243f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:37.018495 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93be7a5-2da7-4a79-8c7a-8b24071a002d None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:37.019001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93be7a5-2da7-4a79-8c7a-8b24071a002d None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:37.019473 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93be7a5-2da7-4a79-8c7a-8b24071a002d None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:37.021284 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 124/236] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:48:37.026919 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6d2d6-2a36-4a2e-8763-f2f70a9e51bc None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:37.027061 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6d2d6-2a36-4a2e-8763-f2f70a9e51bc None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:37.027316 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6d2d6-2a36-4a2e-8763-f2f70a9e51bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:37.042782 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c6bd1655-cf5a-4b15-958a-8baac140243f None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:37.045012 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c6bd1655-cf5a-4b15-958a-8baac140243f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:37.065919 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d0f6d2d6-2a36-4a2e-8763-f2f70a9e51bc None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:37.068834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d0f6d2d6-2a36-4a2e-8763-f2f70a9e51bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:37.182643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 125/237] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:48:37.187122 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 113/238] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:37.209307 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6d2d6-2a36-4a2e-8763-f2f70a9e51bc None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:37.242793 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd1655-cf5a-4b15-958a-8baac140243f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:37.328604 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6d2d6-2a36-4a2e-8763-f2f70a9e51bc None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:37.373350 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa1020-071e-40f7-bd0a-0c5e63a8e925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:37.374163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa1020-071e-40f7-bd0a-0c5e63a8e925 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:37.374443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa1020-071e-40f7-bd0a-0c5e63a8e925 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:37.374717 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa1020-071e-40f7-bd0a-0c5e63a8e925 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:37.381749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa1020-071e-40f7-bd0a-0c5e63a8e925 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:37.383795 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa1020-071e-40f7-bd0a-0c5e63a8e925 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:37.389248 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 126/239] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:48:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:37.390772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa1020-071e-40f7-bd0a-0c5e63a8e925 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:37.409732 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd1655-cf5a-4b15-958a-8baac140243f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:37.463338 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3665a72-85eb-42d4-9fa2-bb9f742c99b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:37.464128 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3665a72-85eb-42d4-9fa2-bb9f742c99b1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:37.464403 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3665a72-85eb-42d4-9fa2-bb9f742c99b1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:37.464673 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3665a72-85eb-42d4-9fa2-bb9f742c99b1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:37.480934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3665a72-85eb-42d4-9fa2-bb9f742c99b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:37.488856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-d3665a72-85eb-42d4-9fa2-bb9f742c99b1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 22:48:37.488856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:37.488856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:37.488856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:37.488856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:37.488856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:37.488856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:37.488856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:37.488856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:37.488856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:37.488856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:37.488856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:37.488856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:37.488856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:37.488856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:37.488856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:37.489728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:37.489728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:37.489728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:37.489728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:37.489728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:37.489728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:37.489728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:37.489728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:37.489728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:37.489728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:37.489728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:37.489728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:48:37.489728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:37.489728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:37.489728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:37.489728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:37.490962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:37.490962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:37.490962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:37.490962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:37.490962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:37.490962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:37.490962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:37.490962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:37.490962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:37.490962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:37.490962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 22:48:37.490962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:48:37.490962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 114/240] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:48:37.504047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3665a72-85eb-42d4-9fa2-bb9f742c99b1 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:37.588632 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa1020-071e-40f7-bd0a-0c5e63a8e925 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:37.676584 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3665a72-85eb-42d4-9fa2-bb9f742c99b1 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:37.699393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31b19b-0fdb-4110-a273-ba0f10add5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:37.700076 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31b19b-0fdb-4110-a273-ba0f10add5af None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:37.700355 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31b19b-0fdb-4110-a273-ba0f10add5af None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:37.700627 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31b19b-0fdb-4110-a273-ba0f10add5af None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:37.701341 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31b19b-0fdb-4110-a273-ba0f10add5af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:37.702888 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31b19b-0fdb-4110-a273-ba0f10add5af None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:37.722230 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be48bfc-1ac3-4b84-845c-aceb97c607b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:37.722910 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be48bfc-1ac3-4b84-845c-aceb97c607b8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:37.723185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be48bfc-1ac3-4b84-845c-aceb97c607b8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:37.723455 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be48bfc-1ac3-4b84-845c-aceb97c607b8 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:37.724314 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be48bfc-1ac3-4b84-845c-aceb97c607b8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:37.724618 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be48bfc-1ac3-4b84-845c-aceb97c607b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:37.726895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be48bfc-1ac3-4b84-845c-aceb97c607b8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:37.740075 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 127/241] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 9 22:48:37 2022] POST /identity/v3/users => generated 293 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:48:37.743870 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 115/242] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:37.749648 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31b19b-0fdb-4110-a273-ba0f10add5af None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:37.857135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31b19b-0fdb-4110-a273-ba0f10add5af None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:37.894032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964024e0-93de-461d-8ab7-c49f307d7a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:37.894952 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964024e0-93de-461d-8ab7-c49f307d7a21 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:37.895229 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964024e0-93de-461d-8ab7-c49f307d7a21 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:37.895501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964024e0-93de-461d-8ab7-c49f307d7a21 None admin] PATH_INFO: `/v3/users/a8b11af91c5045b3bc9ffbed69882fbc` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:37.907042 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e9d42-dec7-497d-824a-4b56a9a61cff None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:37.907404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e9d42-dec7-497d-824a-4b56a9a61cff None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:37.907595 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e9d42-dec7-497d-824a-4b56a9a61cff None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:37.909319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 116/243] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:37.915548 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837445e2-08ef-4057-80e8-bb501ab30b09 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:37.915856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837445e2-08ef-4057-80e8-bb501ab30b09 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:37.916098 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837445e2-08ef-4057-80e8-bb501ab30b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:37.916573 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964024e0-93de-461d-8ab7-c49f307d7a21 None admin] RBAC: Authorizing `identity:get_user(user_id=a8b11af91c5045b3bc9ffbed69882fbc)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:37.918035 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964024e0-93de-461d-8ab7-c49f307d7a21 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:37.920051 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 128/244] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:37 2022] GET /identity/v3/users/a8b11af91c5045b3bc9ffbed69882fbc => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:37.926356 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964024e0-93de-461d-8ab7-c49f307d7a21 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:37.941028 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-837445e2-08ef-4057-80e8-bb501ab30b09 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:37.942684 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-837445e2-08ef-4057-80e8-bb501ab30b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:38.046754 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 117/245] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:38.073729 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837445e2-08ef-4057-80e8-bb501ab30b09 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:38.074175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964024e0-93de-461d-8ab7-c49f307d7a21 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:38.114750 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba6dd1-d7d1-485a-9172-db0c80cb78e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:38.115338 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba6dd1-d7d1-485a-9172-db0c80cb78e0 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:38.115619 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba6dd1-d7d1-485a-9172-db0c80cb78e0 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:38.115894 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba6dd1-d7d1-485a-9172-db0c80cb78e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:38.188093 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837445e2-08ef-4057-80e8-bb501ab30b09 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:38.202288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba6dd1-d7d1-485a-9172-db0c80cb78e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:38.203776 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba6dd1-d7d1-485a-9172-db0c80cb78e0 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:38.233462 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb874f7-4316-4eb3-9163-eeba0ea18667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:38.234307 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb874f7-4316-4eb3-9163-eeba0ea18667 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:38.234849 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb874f7-4316-4eb3-9163-eeba0ea18667 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:38.235123 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb874f7-4316-4eb3-9163-eeba0ea18667 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:38.249425 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb874f7-4316-4eb3-9163-eeba0ea18667 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:38.259947 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-9eb874f7-4316-4eb3-9163-eeba0ea18667 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:38.259947 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:38.259947 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:38.259947 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:38.259947 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:38.259947 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:38.259947 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:38.259947 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:38.259947 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:38.259947 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:38.259947 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:38.259947 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:38.259947 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:38.259947 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:38.259947 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:38.259947 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:38.260934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:38.260934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:38.260934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:38.260934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:38.260934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:38.260934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:38.260934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:38.260934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:38.260934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:38.260934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:38.260934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:38.260934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:48:38.260934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:38.260934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:38.260934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:38.260934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:38.261748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:38.261748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:38.261748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:38.261748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:38.261748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:38.261748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:38.261748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:38.261748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:38.261748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:38.261748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:38.261748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:38.261748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:48:38.265237 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 118/246] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:48:38.271003 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb874f7-4316-4eb3-9163-eeba0ea18667 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:38.330087 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 129/247] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:48:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 22:48:38.336396 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba6dd1-d7d1-485a-9172-db0c80cb78e0 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:38.388528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb874f7-4316-4eb3-9163-eeba0ea18667 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:38.426023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e6d6b-653d-477a-8aa5-da43f4759936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:38.429468 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e6d6b-653d-477a-8aa5-da43f4759936 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:38.429468 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e6d6b-653d-477a-8aa5-da43f4759936 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:38.429468 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e6d6b-653d-477a-8aa5-da43f4759936 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:38.429468 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e6d6b-653d-477a-8aa5-da43f4759936 None admin] RBAC: Adding query filter params (name=member) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:38.429468 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e6d6b-653d-477a-8aa5-da43f4759936 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:38.430157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e6d6b-653d-477a-8aa5-da43f4759936 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:38.443462 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 119/248] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:38.449561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e6d6b-653d-477a-8aa5-da43f4759936 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:38.452371 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba6dd1-d7d1-485a-9172-db0c80cb78e0 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:38.499009 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611b014-81b9-49b3-970b-863e57560264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:38.499876 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611b014-81b9-49b3-970b-863e57560264 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:38.500156 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611b014-81b9-49b3-970b-863e57560264 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:38.500426 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611b014-81b9-49b3-970b-863e57560264 None admin] PATH_INFO: `/v3/projects/db7334eab91d41fa884c491d1364a253` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:38.501610 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0611b014-81b9-49b3-970b-863e57560264 None admin] RBAC: Authorizing `identity:get_project(project_id=db7334eab91d41fa884c491d1364a253)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:38.503273 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0611b014-81b9-49b3-970b-863e57560264 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:38.504799 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 130/249] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:38 2022] GET /identity/v3/projects/db7334eab91d41fa884c491d1364a253 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:38.505983 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0bd7b-bf3d-4fa5-9c1d-2fb19ca4a9c4 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:38.506282 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0bd7b-bf3d-4fa5-9c1d-2fb19ca4a9c4 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:38.506554 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0bd7b-bf3d-4fa5-9c1d-2fb19ca4a9c4 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:38.508445 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 131/250] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:38.513334 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0bd7b-bf3d-4fa5-9c1d-2fb19ca4a9c4 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:38.574846 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e6d6b-653d-477a-8aa5-da43f4759936 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:38.610809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54596284-1a14-43e9-9919-b2eca3ca2c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:38.612039 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54596284-1a14-43e9-9919-b2eca3ca2c60 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:38.612538 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54596284-1a14-43e9-9919-b2eca3ca2c60 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:38.613023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54596284-1a14-43e9-9919-b2eca3ca2c60 None admin] PATH_INFO: `/v3/users/08f98fc5803d4101a40d48ae3945a058` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:38.614476 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54596284-1a14-43e9-9919-b2eca3ca2c60 None admin] RBAC: Authorizing `identity:get_user(user_id=08f98fc5803d4101a40d48ae3945a058)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:38.616156 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54596284-1a14-43e9-9919-b2eca3ca2c60 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:38.618333 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 120/251] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:38 2022] GET /identity/v3/users/08f98fc5803d4101a40d48ae3945a058 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:38.619978 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b2d92-7cb6-405b-bc40-cc1ae384b001 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:38.620361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b2d92-7cb6-405b-bc40-cc1ae384b001 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:38.620733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b2d92-7cb6-405b-bc40-cc1ae384b001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:38.621049 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0bd7b-bf3d-4fa5-9c1d-2fb19ca4a9c4 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:38.664007 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-df0b2d92-7cb6-405b-bc40-cc1ae384b001 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:38.666177 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-df0b2d92-7cb6-405b-bc40-cc1ae384b001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:38.672062 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d5187-5a25-4640-a3ba-4c6161a0eeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:38.673143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d5187-5a25-4640-a3ba-4c6161a0eeeb None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:38.673621 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d5187-5a25-4640-a3ba-4c6161a0eeeb None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:38.674147 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d5187-5a25-4640-a3ba-4c6161a0eeeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:38.675263 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d5187-5a25-4640-a3ba-4c6161a0eeeb None admin] RBAC: Adding query filter params (role.id=949d4917fcdb41e2a938aadb277d1970, scope.project.id=db7334eab91d41fa884c491d1364a253, user.id=a8b11af91c5045b3bc9ffbed69882fbc) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:38.675785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d5187-5a25-4640-a3ba-4c6161a0eeeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:38.679961 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d5187-5a25-4640-a3ba-4c6161a0eeeb None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:38.692139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 132/252] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:48:38 2022] GET /identity/v3/role_assignments?user.id=a8b11af91c5045b3bc9ffbed69882fbc&scope.project.id=db7334eab91d41fa884c491d1364a253&role.id=949d4917fcdb41e2a938aadb277d1970 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:38.693310 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d5187-5a25-4640-a3ba-4c6161a0eeeb None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:38.811927 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 121/253] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:38.855452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de40744-ee10-4ac2-8a58-ade23a49cdfc None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:38.856078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de40744-ee10-4ac2-8a58-ade23a49cdfc None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:38.856549 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de40744-ee10-4ac2-8a58-ade23a49cdfc None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:38.858480 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 122/254] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:38.872182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fc564-58dc-408e-a92d-3b16e3731ef6 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:38.872788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fc564-58dc-408e-a92d-3b16e3731ef6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:38.873258 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fc564-58dc-408e-a92d-3b16e3731ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:38.877037 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d5187-5a25-4640-a3ba-4c6161a0eeeb None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:38.894242 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-757fc564-58dc-408e-a92d-3b16e3731ef6 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:38.898339 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-757fc564-58dc-408e-a92d-3b16e3731ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:38.952902 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83af5e-0194-44a9-b042-c6484f477609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:38.953438 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83af5e-0194-44a9-b042-c6484f477609 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:38.953715 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83af5e-0194-44a9-b042-c6484f477609 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:38.953984 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83af5e-0194-44a9-b042-c6484f477609 None admin] PATH_INFO: `/v3/projects/db7334eab91d41fa884c491d1364a253` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:38.958941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83af5e-0194-44a9-b042-c6484f477609 None admin] RBAC: Authorizing `identity:get_project(project_id=db7334eab91d41fa884c491d1364a253)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:38.960503 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83af5e-0194-44a9-b042-c6484f477609 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:38.962471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 133/255] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:38 2022] GET /identity/v3/projects/db7334eab91d41fa884c491d1364a253 => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:38.963158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83af5e-0194-44a9-b042-c6484f477609 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:39.011182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 123/256] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:48:39.012415 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fc564-58dc-408e-a92d-3b16e3731ef6 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:39.096629 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83af5e-0194-44a9-b042-c6484f477609 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:39.130894 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591a09e-1a2a-4b4e-88cc-8211c57a8cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:39.131694 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591a09e-1a2a-4b4e-88cc-8211c57a8cd2 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:39.132031 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591a09e-1a2a-4b4e-88cc-8211c57a8cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:39.132277 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591a09e-1a2a-4b4e-88cc-8211c57a8cd2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:39.134431 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fc564-58dc-408e-a92d-3b16e3731ef6 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:39.142961 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6591a09e-1a2a-4b4e-88cc-8211c57a8cd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:39.151251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-6591a09e-1a2a-4b4e-88cc-8211c57a8cd2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:48:39.151251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:39.151251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:39.151251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:39.151251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:39.151251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:39.151251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:39.151251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:39.151251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:39.151251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:39.151251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:39.151251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:39.151251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:39.151251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:39.151251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:39.151251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:39.152403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:39.152403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:39.152403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:39.152403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:39.152403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:39.152403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:39.152403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:39.152403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:39.152403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:39.152403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:39.152403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:39.152403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:39.152403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:39.152403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:39.152403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:39.152403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:39.153380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:39.153380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:39.153380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:39.153380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:39.153380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:39.153380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:39.153380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:39.153380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:39.153380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:39.153380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:39.153380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:48:39.153380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:39.159805 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 134/257] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:48:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:39.160777 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591a09e-1a2a-4b4e-88cc-8211c57a8cd2 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:39.176040 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56098c10-014c-4274-8183-280c8c0d164d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:39.177638 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56098c10-014c-4274-8183-280c8c0d164d None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:39.178067 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56098c10-014c-4274-8183-280c8c0d164d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:39.179360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56098c10-014c-4274-8183-280c8c0d164d None admin] PATH_INFO: `/v3/projects/db7334eab91d41fa884c491d1364a253/users/08f98fc5803d4101a40d48ae3945a058/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:39.181892 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56098c10-014c-4274-8183-280c8c0d164d None admin] RBAC: Authorizing `identity:create_grant(project_id=db7334eab91d41fa884c491d1364a253, user_id=08f98fc5803d4101a40d48ae3945a058, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:39.183527 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56098c10-014c-4274-8183-280c8c0d164d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:39.213276 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 124/258] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:48:39 2022] PUT /identity/v3/projects/db7334eab91d41fa884c491d1364a253/users/08f98fc5803d4101a40d48ae3945a058/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:48:39.214432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56098c10-014c-4274-8183-280c8c0d164d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:39.306589 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591a09e-1a2a-4b4e-88cc-8211c57a8cd2 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:39.343223 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe351c9-92c5-4b0f-9de9-5c5003c4ed42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:39.344207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe351c9-92c5-4b0f-9de9-5c5003c4ed42 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:39.344604 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe351c9-92c5-4b0f-9de9-5c5003c4ed42 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:39.344973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe351c9-92c5-4b0f-9de9-5c5003c4ed42 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:39.359132 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe351c9-92c5-4b0f-9de9-5c5003c4ed42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:39.369135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-dbe351c9-92c5-4b0f-9de9-5c5003c4ed42 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:48:39.369135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:39.369135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:39.369135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:39.369135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:39.369135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:39.369135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:39.369135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:39.369135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:39.369135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:39.369135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:39.369135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:39.369135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:39.369135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:39.369135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:39.369135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:39.370097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:39.370097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:39.370097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:39.370097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:39.370097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:39.370097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:39.370097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:39.370097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:39.370097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:39.370097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:39.370097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:39.370097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:39.370097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:39.370097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:39.370097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:39.370097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:39.371024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:39.371024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:39.371024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:39.371024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:39.371024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:39.371024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:39.371024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:39.371024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:39.371024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:39.371024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:39.371024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:48:39.371024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:39.371024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 135/259] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 9 22:48:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:39.376762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe351c9-92c5-4b0f-9de9-5c5003c4ed42 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:39.468198 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56098c10-014c-4274-8183-280c8c0d164d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:39.503104 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e2218-6e1f-4c13-b6ed-a899dd87b4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:39.503786 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe351c9-92c5-4b0f-9de9-5c5003c4ed42 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:39.504389 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e2218-6e1f-4c13-b6ed-a899dd87b4d1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:39.504878 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e2218-6e1f-4c13-b6ed-a899dd87b4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:39.505347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e2218-6e1f-4c13-b6ed-a899dd87b4d1 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:39.506329 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e2218-6e1f-4c13-b6ed-a899dd87b4d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:39.506878 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e2218-6e1f-4c13-b6ed-a899dd87b4d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:39.508499 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e2218-6e1f-4c13-b6ed-a899dd87b4d1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:39.518196 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 125/260] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:39.524329 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e2218-6e1f-4c13-b6ed-a899dd87b4d1 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:39.584050 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad552a43-49e4-4ba8-86ab-f398bc802e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:39.584722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad552a43-49e4-4ba8-86ab-f398bc802e25 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:39.585682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad552a43-49e4-4ba8-86ab-f398bc802e25 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:39.585959 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad552a43-49e4-4ba8-86ab-f398bc802e25 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:39.587063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad552a43-49e4-4ba8-86ab-f398bc802e25 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:39.587371 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad552a43-49e4-4ba8-86ab-f398bc802e25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:39.589122 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad552a43-49e4-4ba8-86ab-f398bc802e25 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:39.598883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 136/261] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:48:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:39.607199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad552a43-49e4-4ba8-86ab-f398bc802e25 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:39.666632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e2218-6e1f-4c13-b6ed-a899dd87b4d1 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:39.701323 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09daef6e-d8e0-4366-9418-e42449e48120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:39.702285 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09daef6e-d8e0-4366-9418-e42449e48120 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:39.702566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09daef6e-d8e0-4366-9418-e42449e48120 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:39.702840 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09daef6e-d8e0-4366-9418-e42449e48120 None admin] PATH_INFO: `/v3/users/31c77c49b318466f8c82caa90facbf35` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:39.704063 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09daef6e-d8e0-4366-9418-e42449e48120 None admin] RBAC: Authorizing `identity:get_user(user_id=31c77c49b318466f8c82caa90facbf35)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:39.705517 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09daef6e-d8e0-4366-9418-e42449e48120 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:39.707487 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 126/262] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:39 2022] GET /identity/v3/users/31c77c49b318466f8c82caa90facbf35 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:39.713718 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09daef6e-d8e0-4366-9418-e42449e48120 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:39.825487 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09daef6e-d8e0-4366-9418-e42449e48120 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:39.861698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808b59b-9367-4142-ab05-de4ecc7944cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:39.862394 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808b59b-9367-4142-ab05-de4ecc7944cc None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:39.862675 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808b59b-9367-4142-ab05-de4ecc7944cc None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:39.863279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808b59b-9367-4142-ab05-de4ecc7944cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:39.868739 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad552a43-49e4-4ba8-86ab-f398bc802e25 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:39.939101 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9808b59b-9367-4142-ab05-de4ecc7944cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:39.940788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9808b59b-9367-4142-ab05-de4ecc7944cc None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:39.963417 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1962d4b-943a-4e31-8bf6-bfba959e0555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:39.964666 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1962d4b-943a-4e31-8bf6-bfba959e0555 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:39.965148 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1962d4b-943a-4e31-8bf6-bfba959e0555 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:39.965630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1962d4b-943a-4e31-8bf6-bfba959e0555 None admin] PATH_INFO: `/v3/users/31c77c49b318466f8c82caa90facbf35` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:39.970860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1962d4b-943a-4e31-8bf6-bfba959e0555 None admin] RBAC: Authorizing `identity:get_user(user_id=31c77c49b318466f8c82caa90facbf35)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:39.972551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1962d4b-943a-4e31-8bf6-bfba959e0555 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:39.974802 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 137/263] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:39 2022] GET /identity/v3/users/31c77c49b318466f8c82caa90facbf35 => generated 283 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:39.980529 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1962d4b-943a-4e31-8bf6-bfba959e0555 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:40.049403 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 127/264] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:48:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 22:48:40.055748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808b59b-9367-4142-ab05-de4ecc7944cc None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:40.192555 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1962d4b-943a-4e31-8bf6-bfba959e0555 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:40.248190 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee06f03-089e-47dc-87f9-5293d8869960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:40.248849 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee06f03-089e-47dc-87f9-5293d8869960 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:40.249134 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee06f03-089e-47dc-87f9-5293d8869960 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:40.249409 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee06f03-089e-47dc-87f9-5293d8869960 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:40.292037 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808b59b-9367-4142-ab05-de4ecc7944cc None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:40.363505 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee06f03-089e-47dc-87f9-5293d8869960 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:40.365166 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee06f03-089e-47dc-87f9-5293d8869960 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:40.386837 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c7d8a-9984-44bc-acaa-7e95a7ddf308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:40.388046 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c7d8a-9984-44bc-acaa-7e95a7ddf308 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:40.388554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c7d8a-9984-44bc-acaa-7e95a7ddf308 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:40.389031 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c7d8a-9984-44bc-acaa-7e95a7ddf308 None admin] PATH_INFO: `/v3/projects/db7334eab91d41fa884c491d1364a253` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:40.390507 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c7d8a-9984-44bc-acaa-7e95a7ddf308 None admin] RBAC: Authorizing `identity:get_project(project_id=db7334eab91d41fa884c491d1364a253)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:40.392168 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c7d8a-9984-44bc-acaa-7e95a7ddf308 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:40.398277 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 128/265] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:40 2022] GET /identity/v3/projects/db7334eab91d41fa884c491d1364a253 => generated 302 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:40.403066 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c7d8a-9984-44bc-acaa-7e95a7ddf308 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:40.474342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 138/266] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:48:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 22:48:40.487934 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee06f03-089e-47dc-87f9-5293d8869960 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:40.598805 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee06f03-089e-47dc-87f9-5293d8869960 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:40.633185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c7d8a-9984-44bc-acaa-7e95a7ddf308 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:40.646139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77932750-2204-4ab0-809d-7f669d464413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:40.646139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77932750-2204-4ab0-809d-7f669d464413 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:40.646139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77932750-2204-4ab0-809d-7f669d464413 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:40.646139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77932750-2204-4ab0-809d-7f669d464413 None admin] PATH_INFO: `/v3/projects/db7334eab91d41fa884c491d1364a253` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:40.646491 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77932750-2204-4ab0-809d-7f669d464413 None admin] RBAC: Authorizing `identity:get_project(project_id=db7334eab91d41fa884c491d1364a253)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:40.646491 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77932750-2204-4ab0-809d-7f669d464413 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:40.648346 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 139/267] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:40 2022] GET /identity/v3/projects/db7334eab91d41fa884c491d1364a253 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:40.654494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77932750-2204-4ab0-809d-7f669d464413 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:40.697309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951ee12-7b23-4c94-b058-07bae411cae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:40.697978 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951ee12-7b23-4c94-b058-07bae411cae9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:40.698285 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951ee12-7b23-4c94-b058-07bae411cae9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:40.698569 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951ee12-7b23-4c94-b058-07bae411cae9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:40.699407 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951ee12-7b23-4c94-b058-07bae411cae9 None admin] RBAC: Adding query filter params (role.id=bda550492f4f4c5c83f52fa89fc18a7d, scope.project.id=db7334eab91d41fa884c491d1364a253, user.id=31c77c49b318466f8c82caa90facbf35) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:40.699717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951ee12-7b23-4c94-b058-07bae411cae9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:40.701168 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951ee12-7b23-4c94-b058-07bae411cae9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:40.720211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 129/268] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:48:40 2022] GET /identity/v3/role_assignments?user.id=31c77c49b318466f8c82caa90facbf35&scope.project.id=db7334eab91d41fa884c491d1364a253&role.id=bda550492f4f4c5c83f52fa89fc18a7d => generated 608 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:40.809636 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77932750-2204-4ab0-809d-7f669d464413 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:40.857548 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b15e2-0f75-46a8-b984-5d091ca2aa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:40.858230 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b15e2-0f75-46a8-b984-5d091ca2aa7c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:40.858515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b15e2-0f75-46a8-b984-5d091ca2aa7c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:40.858790 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b15e2-0f75-46a8-b984-5d091ca2aa7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:40.859890 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b15e2-0f75-46a8-b984-5d091ca2aa7c None admin] RBAC: Adding query filter params (role.id=46ddc3cf4b124399a275bc639635e10a, scope.project.id=db7334eab91d41fa884c491d1364a253, user.id=31c77c49b318466f8c82caa90facbf35) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:40.860207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b15e2-0f75-46a8-b984-5d091ca2aa7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:40.861645 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b15e2-0f75-46a8-b984-5d091ca2aa7c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:40.871219 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 140/269] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:48:40 2022] GET /identity/v3/role_assignments?user.id=31c77c49b318466f8c82caa90facbf35&scope.project.id=db7334eab91d41fa884c491d1364a253&role.id=46ddc3cf4b124399a275bc639635e10a => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:41.213190 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029d6f0-b21d-4f19-a992-5e7492c9c48a None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:41.213525 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029d6f0-b21d-4f19-a992-5e7492c9c48a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:41.213703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029d6f0-b21d-4f19-a992-5e7492c9c48a None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:41.215076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 130/270] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:41.220691 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23b66e-0ca1-4a53-b847-1e0b2f7d733f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:41.221185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23b66e-0ca1-4a53-b847-1e0b2f7d733f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:41.221283 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23b66e-0ca1-4a53-b847-1e0b2f7d733f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:41.243345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-2c23b66e-0ca1-4a53-b847-1e0b2f7d733f None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:41.244573 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2c23b66e-0ca1-4a53-b847-1e0b2f7d733f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:41.323345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 141/271] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:41.348554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029d6f0-b21d-4f19-a992-5e7492c9c48a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:41.424861 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d756a0-1829-4210-bc85-020860535f39 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:41.425181 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d756a0-1829-4210-bc85-020860535f39 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:41.425427 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d756a0-1829-4210-bc85-020860535f39 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:41.427199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 142/272] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:41.441206 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635c5cc-8a09-4d42-b9d1-9c9672f46c8a None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:41.441646 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635c5cc-8a09-4d42-b9d1-9c9672f46c8a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:41.442030 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635c5cc-8a09-4d42-b9d1-9c9672f46c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:41.474702 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-1635c5cc-8a09-4d42-b9d1-9c9672f46c8a None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:41.475839 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1635c5cc-8a09-4d42-b9d1-9c9672f46c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:41.487045 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029d6f0-b21d-4f19-a992-5e7492c9c48a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:41.527903 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dd7fb-dd1b-48b5-bb32-9f70aee48c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:41.529062 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dd7fb-dd1b-48b5-bb32-9f70aee48c8f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:41.529549 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dd7fb-dd1b-48b5-bb32-9f70aee48c8f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:41.530066 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dd7fb-dd1b-48b5-bb32-9f70aee48c8f None admin] PATH_INFO: `/v3/roles/member` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:41.541613 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12dd7fb-dd1b-48b5-bb32-9f70aee48c8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:41.549816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-b12dd7fb-dd1b-48b5-bb32-9f70aee48c8f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:41.549816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:41.549816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:41.549816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:41.549816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:41.549816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:41.549816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:41.549816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:41.549816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:41.549816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:41.549816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:41.549816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:41.549816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:41.549816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:41.549816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:41.549816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:41.550733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:41.550733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:41.550733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:41.550733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:41.550733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:41.550733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:41.550733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:41.550733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:41.550733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:41.550733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:41.550733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:41.550733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:48:41.550733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:41.550733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:41.550733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:41.550733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:41.551565 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:41.551565 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:41.551565 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:41.551565 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:41.551565 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:41.551565 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:41.551565 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:41.551565 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:41.551565 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:41.551565 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:41.551565 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:41.551565 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:48:41.554274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 131/273] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:41.560280 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dd7fb-dd1b-48b5-bb32-9f70aee48c8f None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:41.566193 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 143/274] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:48:41.606021 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635c5cc-8a09-4d42-b9d1-9c9672f46c8a None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:41.705735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dd7fb-dd1b-48b5-bb32-9f70aee48c8f None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:41.735284 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635c5cc-8a09-4d42-b9d1-9c9672f46c8a None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:41.748877 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566cba51-61af-42a8-ad65-67f027e0d2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:41.749964 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cba51-61af-42a8-ad65-67f027e0d2d0 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:41.750479 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cba51-61af-42a8-ad65-67f027e0d2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:41.750954 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cba51-61af-42a8-ad65-67f027e0d2d0 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:41.751970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566cba51-61af-42a8-ad65-67f027e0d2d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:41.752489 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566cba51-61af-42a8-ad65-67f027e0d2d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:41.754075 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566cba51-61af-42a8-ad65-67f027e0d2d0 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:41.763123 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 132/275] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:41.768717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566cba51-61af-42a8-ad65-67f027e0d2d0 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:41.774890 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cb6e0-a69e-4196-a465-815dfa889934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:41.775688 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cb6e0-a69e-4196-a465-815dfa889934 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:41.775970 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cb6e0-a69e-4196-a465-815dfa889934 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:41.776244 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cb6e0-a69e-4196-a465-815dfa889934 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:41.777415 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322cb6e0-a69e-4196-a465-815dfa889934 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:41.778912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322cb6e0-a69e-4196-a465-815dfa889934 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:41.781023 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 144/276] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:48:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:41.786553 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cb6e0-a69e-4196-a465-815dfa889934 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:41.872785 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566cba51-61af-42a8-ad65-67f027e0d2d0 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:41.886510 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cb6e0-a69e-4196-a465-815dfa889934 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:41.906654 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b6c80-63fe-4d83-a8fd-7a4a2acdbad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:41.907522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b6c80-63fe-4d83-a8fd-7a4a2acdbad9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:41.907800 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b6c80-63fe-4d83-a8fd-7a4a2acdbad9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:41.908071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b6c80-63fe-4d83-a8fd-7a4a2acdbad9 None admin] PATH_INFO: `/v3/users/b91f416e19a5463e8b18d6e052634c41` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:41.922342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f76c6c1-57ed-46ed-a26b-22929d35e1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:41.923106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76c6c1-57ed-46ed-a26b-22929d35e1a4 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:41.923387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76c6c1-57ed-46ed-a26b-22929d35e1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:41.923659 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76c6c1-57ed-46ed-a26b-22929d35e1a4 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:41.923835 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b6c80-63fe-4d83-a8fd-7a4a2acdbad9 None admin] RBAC: Authorizing `identity:get_user(user_id=b91f416e19a5463e8b18d6e052634c41)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:41.924358 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f76c6c1-57ed-46ed-a26b-22929d35e1a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:41.925290 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b6c80-63fe-4d83-a8fd-7a4a2acdbad9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:41.925774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f76c6c1-57ed-46ed-a26b-22929d35e1a4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:41.932319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 133/277] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:41 2022] GET /identity/v3/users/b91f416e19a5463e8b18d6e052634c41 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:41.932319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60b59f-f308-42c0-96ad-036af3ab7f18 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:41.932319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60b59f-f308-42c0-96ad-036af3ab7f18 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:41.932319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60b59f-f308-42c0-96ad-036af3ab7f18 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:41.934242 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 134/278] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:41.935328 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60b59f-f308-42c0-96ad-036af3ab7f18 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:41.949738 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 145/279] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 9 22:48:41 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:48:41.951358 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e2616-4c73-48fe-8dca-d128b1c7f97c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:41.951856 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e2616-4c73-48fe-8dca-d128b1c7f97c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:41.952720 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e2616-4c73-48fe-8dca-d128b1c7f97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:41.973832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ea5e2616-4c73-48fe-8dca-d128b1c7f97c None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:41.976141 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ea5e2616-4c73-48fe-8dca-d128b1c7f97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:42.056185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60b59f-f308-42c0-96ad-036af3ab7f18 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:42.070378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 146/280] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:42.092770 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e2616-4c73-48fe-8dca-d128b1c7f97c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:42.104015 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1560e5a-8227-489a-bf69-f9d548ac4821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:42.105077 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1560e5a-8227-489a-bf69-f9d548ac4821 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:42.105562 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1560e5a-8227-489a-bf69-f9d548ac4821 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:42.106056 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1560e5a-8227-489a-bf69-f9d548ac4821 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:42.107109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1560e5a-8227-489a-bf69-f9d548ac4821 None admin] RBAC: Adding query filter params (role.id=3631750e6bf34730aa3557cd90aa6108, user.id=b91f416e19a5463e8b18d6e052634c41, scope.system=all) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:42.107610 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1560e5a-8227-489a-bf69-f9d548ac4821 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:42.109284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1560e5a-8227-489a-bf69-f9d548ac4821 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:42.115658 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 135/281] 10.222.0.23 () {56 vars in 1308 bytes} [Mon May 9 22:48:41 2022] GET /identity/v3/role_assignments?user.id=b91f416e19a5463e8b18d6e052634c41&scope.system=all&role.id=3631750e6bf34730aa3557cd90aa6108 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:42.226555 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e2616-4c73-48fe-8dca-d128b1c7f97c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:42.264909 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60abcb4-4e0f-4782-b506-e6fb9c647a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:42.265722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60abcb4-4e0f-4782-b506-e6fb9c647a49 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:42.266000 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60abcb4-4e0f-4782-b506-e6fb9c647a49 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:42.266323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60abcb4-4e0f-4782-b506-e6fb9c647a49 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:42.278704 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60abcb4-4e0f-4782-b506-e6fb9c647a49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:42.286622 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-e60abcb4-4e0f-4782-b506-e6fb9c647a49 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 22:48:42.286622 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:42.286622 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:42.286622 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:42.286622 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:42.286622 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:42.286622 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:42.286622 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:42.286622 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:42.286622 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:42.286622 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:42.286622 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:42.286622 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:42.286622 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:42.286622 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:42.286622 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:42.287551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:42.287551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:42.287551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:42.287551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:42.287551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:42.287551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:42.287551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:42.287551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:42.287551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:42.287551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:42.287551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:42.287551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:42.287551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:42.287551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:42.287551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:42.287551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:42.288918 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:42.288918 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:42.288918 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:42.288918 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:42.288918 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:42.288918 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:42.288918 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:42.288918 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:42.288918 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:42.288918 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:42.288918 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 22:48:42.288918 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:42.288918 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 147/282] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:48:42.296036 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1560e5a-8227-489a-bf69-f9d548ac4821 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:42.410385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ecaa0-aff5-49f1-8db1-6490030683d2 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:42.410701 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ecaa0-aff5-49f1-8db1-6490030683d2 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:42.410865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ecaa0-aff5-49f1-8db1-6490030683d2 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:42.412231 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 148/283] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:48:42.417350 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1560e5a-8227-489a-bf69-f9d548ac4821 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:42.418918 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce41db-dbb7-4085-913f-0549c5216697 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:42.419206 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce41db-dbb7-4085-913f-0549c5216697 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:42.419482 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce41db-dbb7-4085-913f-0549c5216697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:42.441004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-46ce41db-dbb7-4085-913f-0549c5216697 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:42.442218 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-46ce41db-dbb7-4085-913f-0549c5216697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:42.462026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1c643-4b34-43b4-a548-07f1ea5d9aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:42.463497 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1c643-4b34-43b4-a548-07f1ea5d9aea None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:42.463979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1c643-4b34-43b4-a548-07f1ea5d9aea None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:42.464446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1c643-4b34-43b4-a548-07f1ea5d9aea None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:42.465385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1c643-4b34-43b4-a548-07f1ea5d9aea None admin] RBAC: Adding query filter params (name=reader) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:42.466331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1c643-4b34-43b4-a548-07f1ea5d9aea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:42.468258 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1c643-4b34-43b4-a548-07f1ea5d9aea None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:42.479416 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 136/284] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:42.486487 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1c643-4b34-43b4-a548-07f1ea5d9aea None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:42.526509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 149/285] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:42.567203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce41db-dbb7-4085-913f-0549c5216697 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:42.636240 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1c643-4b34-43b4-a548-07f1ea5d9aea None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:42.676312 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce41db-dbb7-4085-913f-0549c5216697 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:42.704359 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e26618-e7bd-4cce-b77c-7354083826b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:42.705448 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e26618-e7bd-4cce-b77c-7354083826b1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:42.705729 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e26618-e7bd-4cce-b77c-7354083826b1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:42.706003 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e26618-e7bd-4cce-b77c-7354083826b1 None admin] PATH_INFO: `/v3/users/a8b11af91c5045b3bc9ffbed69882fbc` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:42.707497 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e26618-e7bd-4cce-b77c-7354083826b1 None admin] RBAC: Authorizing `identity:get_user(user_id=a8b11af91c5045b3bc9ffbed69882fbc)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:42.708834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e26618-e7bd-4cce-b77c-7354083826b1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:42.711086 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 137/286] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:42 2022] GET /identity/v3/users/a8b11af91c5045b3bc9ffbed69882fbc => generated 297 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:42.717333 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e26618-e7bd-4cce-b77c-7354083826b1 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:42.745097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c30445-7f6e-408a-a723-9f243506983e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:42.745904 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c30445-7f6e-408a-a723-9f243506983e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:42.746206 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c30445-7f6e-408a-a723-9f243506983e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:42.746479 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c30445-7f6e-408a-a723-9f243506983e None admin] PATH_INFO: `/v3/roles/member` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:42.768746 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c30445-7f6e-408a-a723-9f243506983e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:42.779328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-69c30445-7f6e-408a-a723-9f243506983e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:42.779328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:42.779328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:42.779328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:42.779328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:42.779328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:42.779328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:42.779328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:42.779328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:42.779328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:42.779328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:42.779328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:42.779328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:42.779328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:42.779328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:42.779328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:42.780243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:42.780243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:42.780243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:42.780243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:42.780243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:42.780243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:42.780243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:42.780243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:42.780243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:42.780243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:42.780243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:42.780243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:42.780243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:42.780243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:42.780243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:42.780243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:42.781077 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:42.781077 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:42.781077 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:42.781077 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:42.781077 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:42.781077 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:42.781077 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:42.781077 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:42.781077 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:42.781077 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:42.781077 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:42.781077 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:42.781689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 150/287] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:42.789892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c30445-7f6e-408a-a723-9f243506983e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:42.870168 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e26618-e7bd-4cce-b77c-7354083826b1 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:42.904889 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b84d3-d51b-49f7-a025-44e267c05cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:42.905903 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b84d3-d51b-49f7-a025-44e267c05cf8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:42.906030 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b84d3-d51b-49f7-a025-44e267c05cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:42.906323 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b84d3-d51b-49f7-a025-44e267c05cf8 None admin] PATH_INFO: `/v3/projects/db7334eab91d41fa884c491d1364a253` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:42.907788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b84d3-d51b-49f7-a025-44e267c05cf8 None admin] RBAC: Authorizing `identity:get_project(project_id=db7334eab91d41fa884c491d1364a253)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:42.909061 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b84d3-d51b-49f7-a025-44e267c05cf8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:42.911182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 138/288] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:42 2022] GET /identity/v3/projects/db7334eab91d41fa884c491d1364a253 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:42.916200 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b84d3-d51b-49f7-a025-44e267c05cf8 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:42.951243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c30445-7f6e-408a-a723-9f243506983e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:43.043356 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15f2e4-39f9-47e9-82c4-5d2030d2633e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:43.044025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15f2e4-39f9-47e9-82c4-5d2030d2633e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:43.044297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15f2e4-39f9-47e9-82c4-5d2030d2633e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:43.044567 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15f2e4-39f9-47e9-82c4-5d2030d2633e None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:43.045304 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd15f2e4-39f9-47e9-82c4-5d2030d2633e None admin] RBAC: Adding query filter params (name=member) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:43.045602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd15f2e4-39f9-47e9-82c4-5d2030d2633e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:43.045766 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b84d3-d51b-49f7-a025-44e267c05cf8 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:43.046999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd15f2e4-39f9-47e9-82c4-5d2030d2633e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:43.065570 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 151/289] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:43.084485 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15f2e4-39f9-47e9-82c4-5d2030d2633e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:43.087357 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7675ac7c-cfa1-4f84-8856-bc5da226b3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:43.088257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675ac7c-cfa1-4f84-8856-bc5da226b3ef None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:43.088540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675ac7c-cfa1-4f84-8856-bc5da226b3ef None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:43.088817 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675ac7c-cfa1-4f84-8856-bc5da226b3ef None admin] PATH_INFO: `/v3/projects/db7334eab91d41fa884c491d1364a253/users/a8b11af91c5045b3bc9ffbed69882fbc/roles/949d4917fcdb41e2a938aadb277d1970` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:43.090986 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7675ac7c-cfa1-4f84-8856-bc5da226b3ef None admin] RBAC: Authorizing `identity:create_grant(project_id=db7334eab91d41fa884c491d1364a253, user_id=a8b11af91c5045b3bc9ffbed69882fbc, role_id=949d4917fcdb41e2a938aadb277d1970)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:43.092663 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7675ac7c-cfa1-4f84-8856-bc5da226b3ef None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:43.119371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 139/290] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:48:42 2022] PUT /identity/v3/projects/db7334eab91d41fa884c491d1364a253/users/a8b11af91c5045b3bc9ffbed69882fbc/roles/949d4917fcdb41e2a938aadb277d1970 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:48:43.210259 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15f2e4-39f9-47e9-82c4-5d2030d2633e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:43.245762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7c0c1-4d32-4e0f-a8a4-1a9b24f2df82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:43.246745 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7c0c1-4d32-4e0f-a8a4-1a9b24f2df82 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:43.247025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7c0c1-4d32-4e0f-a8a4-1a9b24f2df82 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:43.247297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7c0c1-4d32-4e0f-a8a4-1a9b24f2df82 None admin] PATH_INFO: `/v3/users/08f98fc5803d4101a40d48ae3945a058` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:43.248659 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7c0c1-4d32-4e0f-a8a4-1a9b24f2df82 None admin] RBAC: Authorizing `identity:get_user(user_id=08f98fc5803d4101a40d48ae3945a058)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:43.250107 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7c0c1-4d32-4e0f-a8a4-1a9b24f2df82 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:43.255675 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 152/291] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:43 2022] GET /identity/v3/users/08f98fc5803d4101a40d48ae3945a058 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:43.258515 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7675ac7c-cfa1-4f84-8856-bc5da226b3ef None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:43.387851 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7675ac7c-cfa1-4f84-8856-bc5da226b3ef None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:43.424813 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96376cc7-d6a9-42ff-b2a5-3a8b33454f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:43.425596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96376cc7-d6a9-42ff-b2a5-3a8b33454f26 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:43.425754 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96376cc7-d6a9-42ff-b2a5-3a8b33454f26 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:43.426000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96376cc7-d6a9-42ff-b2a5-3a8b33454f26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:43.495438 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96376cc7-d6a9-42ff-b2a5-3a8b33454f26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:43.496908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96376cc7-d6a9-42ff-b2a5-3a8b33454f26 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:43.598298 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 140/292] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:48:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:48:43.601953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7c0c1-4d32-4e0f-a8a4-1a9b24f2df82 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:43.713701 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7c0c1-4d32-4e0f-a8a4-1a9b24f2df82 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:43.748633 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825a4ee-6d91-4402-8669-3c99f1a7a24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:43.749463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825a4ee-6d91-4402-8669-3c99f1a7a24a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:43.749744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825a4ee-6d91-4402-8669-3c99f1a7a24a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:43.750014 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825a4ee-6d91-4402-8669-3c99f1a7a24a None admin] PATH_INFO: `/v3/projects/db7334eab91d41fa884c491d1364a253` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:43.751244 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825a4ee-6d91-4402-8669-3c99f1a7a24a None admin] RBAC: Authorizing `identity:get_project(project_id=db7334eab91d41fa884c491d1364a253)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:43.752694 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825a4ee-6d91-4402-8669-3c99f1a7a24a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:43.758241 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 153/293] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:43 2022] GET /identity/v3/projects/db7334eab91d41fa884c491d1364a253 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:43.761140 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96376cc7-d6a9-42ff-b2a5-3a8b33454f26 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:43.860836 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96376cc7-d6a9-42ff-b2a5-3a8b33454f26 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:43.895338 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e6d17-3eff-4524-a239-6ee76f757427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:43.895998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e6d17-3eff-4524-a239-6ee76f757427 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:43.896278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e6d17-3eff-4524-a239-6ee76f757427 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:43.896552 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e6d17-3eff-4524-a239-6ee76f757427 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:43.897941 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e6d17-3eff-4524-a239-6ee76f757427 None admin] RBAC: Adding query filter params (role.id=3631750e6bf34730aa3557cd90aa6108, scope.project.id=db7334eab91d41fa884c491d1364a253, user.id=08f98fc5803d4101a40d48ae3945a058) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:43.898283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e6d17-3eff-4524-a239-6ee76f757427 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:43.899720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e6d17-3eff-4524-a239-6ee76f757427 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:43.905884 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 141/294] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:48:43 2022] GET /identity/v3/role_assignments?user.id=08f98fc5803d4101a40d48ae3945a058&scope.project.id=db7334eab91d41fa884c491d1364a253&role.id=3631750e6bf34730aa3557cd90aa6108 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:44.523386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8d330-c002-48bc-a2de-3458026ee93e None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:44.523898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8d330-c002-48bc-a2de-3458026ee93e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:44.523974 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8d330-c002-48bc-a2de-3458026ee93e None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:44.525291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 154/295] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:44.530653 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07d52e-3538-4fe4-8378-1baacc4a22d0 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:44.531072 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07d52e-3538-4fe4-8378-1baacc4a22d0 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:44.531206 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07d52e-3538-4fe4-8378-1baacc4a22d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:44.550067 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-9b07d52e-3538-4fe4-8378-1baacc4a22d0 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:44.551285 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9b07d52e-3538-4fe4-8378-1baacc4a22d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:44.625503 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 142/296] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:44.648509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8d330-c002-48bc-a2de-3458026ee93e None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:44.755005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8d330-c002-48bc-a2de-3458026ee93e None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:44.794866 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059abb73-0fac-42a3-b88a-b662bbdfe970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:44.795598 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059abb73-0fac-42a3-b88a-b662bbdfe970 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:44.795876 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059abb73-0fac-42a3-b88a-b662bbdfe970 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:44.796146 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059abb73-0fac-42a3-b88a-b662bbdfe970 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:44.809116 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059abb73-0fac-42a3-b88a-b662bbdfe970 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:44.817127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-059abb73-0fac-42a3-b88a-b662bbdfe970 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:44.817127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:44.817127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:44.817127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:44.817127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:44.817127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:44.817127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:44.817127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:44.817127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:44.817127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:44.817127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:44.817127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:44.817127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:44.817127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:44.817127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:44.817127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:44.818091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:44.818091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:44.818091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:44.818091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:44.818091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:44.818091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:44.818091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:44.818091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:44.818091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:44.818091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:44.818091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:44.818091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:44.818091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:44.818091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:44.818091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:44.818091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:44.818940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:44.818940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:44.818940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:44.818940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:44.818940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:44.818940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:44.818940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:44.818940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:44.818940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:44.818940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:44.818940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:44.818940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:44.823805 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 155/297] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:44.825243 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07d52e-3538-4fe4-8378-1baacc4a22d0 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:44.925344 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07d52e-3538-4fe4-8378-1baacc4a22d0 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:44.946950 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca1317-d064-475e-942b-916a1aa7ea60 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:44.947266 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca1317-d064-475e-942b-916a1aa7ea60 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:44.947495 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca1317-d064-475e-942b-916a1aa7ea60 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:44.949576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 156/298] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:44.955097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e40fa5-50d3-487a-9f01-fd1a7852e8dc None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:44.955436 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e40fa5-50d3-487a-9f01-fd1a7852e8dc None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:44.955715 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e40fa5-50d3-487a-9f01-fd1a7852e8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:44.969893 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c12cdb-1ce3-4619-ade2-62651ccdaf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:44.970898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c12cdb-1ce3-4619-ade2-62651ccdaf04 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:44.971377 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c12cdb-1ce3-4619-ade2-62651ccdaf04 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:44.971861 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c12cdb-1ce3-4619-ade2-62651ccdaf04 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:44.972529 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d1e40fa5-50d3-487a-9f01-fd1a7852e8dc None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:44.973041 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c12cdb-1ce3-4619-ade2-62651ccdaf04 None admin] RBAC: Adding query filter params (name=member) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:44.973551 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c12cdb-1ce3-4619-ade2-62651ccdaf04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:44.973718 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d1e40fa5-50d3-487a-9f01-fd1a7852e8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:44.975337 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c12cdb-1ce3-4619-ade2-62651ccdaf04 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:44.990438 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 143/299] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:44.992268 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8190b-9278-4bf0-82de-cd29bf1c9256 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:44.992554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8190b-9278-4bf0-82de-cd29bf1c9256 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:44.992827 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8190b-9278-4bf0-82de-cd29bf1c9256 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:44.994256 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 144/300] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:44.994777 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8190b-9278-4bf0-82de-cd29bf1c9256 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:45.046266 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 157/301] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:48:45.046266 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f63d05-ec1c-4c83-b89c-08cb2756728c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:45.046266 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f63d05-ec1c-4c83-b89c-08cb2756728c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:45.046266 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f63d05-ec1c-4c83-b89c-08cb2756728c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:45.068067 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-51f63d05-ec1c-4c83-b89c-08cb2756728c None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:45.069755 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-51f63d05-ec1c-4c83-b89c-08cb2756728c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:45.103255 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8190b-9278-4bf0-82de-cd29bf1c9256 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:45.133482 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 158/302] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:45.134528 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f63d05-ec1c-4c83-b89c-08cb2756728c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:45.146219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e8e36-6bd3-46ac-a892-912d3a54e8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:45.147508 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e8e36-6bd3-46ac-a892-912d3a54e8f8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:45.147964 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e8e36-6bd3-46ac-a892-912d3a54e8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:45.148760 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e8e36-6bd3-46ac-a892-912d3a54e8f8 None admin] PATH_INFO: `/v3/users/b91f416e19a5463e8b18d6e052634c41` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:45.150442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e8e36-6bd3-46ac-a892-912d3a54e8f8 None admin] RBAC: Authorizing `identity:get_user(user_id=b91f416e19a5463e8b18d6e052634c41)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:45.152513 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e8e36-6bd3-46ac-a892-912d3a54e8f8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:45.154773 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 145/303] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:44 2022] GET /identity/v3/users/b91f416e19a5463e8b18d6e052634c41 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:45.159053 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e8e36-6bd3-46ac-a892-912d3a54e8f8 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:45.233590 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f63d05-ec1c-4c83-b89c-08cb2756728c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:45.253958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e8e36-6bd3-46ac-a892-912d3a54e8f8 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:45.266382 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ecd65-99ff-41bf-b86b-e131e73955c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:45.267169 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ecd65-99ff-41bf-b86b-e131e73955c9 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:45.267445 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ecd65-99ff-41bf-b86b-e131e73955c9 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:45.267704 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ecd65-99ff-41bf-b86b-e131e73955c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:45.268847 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ecd65-99ff-41bf-b86b-e131e73955c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:45.270313 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ecd65-99ff-41bf-b86b-e131e73955c9 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:45.276837 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 159/304] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:48:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:45.276837 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ecd65-99ff-41bf-b86b-e131e73955c9 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:45.291801 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8453c08-3cb5-4b20-9e49-6c536e78fc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:45.292662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8453c08-3cb5-4b20-9e49-6c536e78fc47 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:45.292945 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8453c08-3cb5-4b20-9e49-6c536e78fc47 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:45.293216 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8453c08-3cb5-4b20-9e49-6c536e78fc47 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:45.303355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8453c08-3cb5-4b20-9e49-6c536e78fc47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:45.311490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-f8453c08-3cb5-4b20-9e49-6c536e78fc47 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 22:48:45.311490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:45.311490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:45.311490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:45.311490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:45.311490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:45.311490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:45.311490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:45.311490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:45.311490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:45.311490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:45.311490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:45.311490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:45.311490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:45.311490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:45.311490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:45.312408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:45.312408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:45.312408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:45.312408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:45.312408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:45.312408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:45.312408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:45.312408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:45.312408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:45.312408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:45.312408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:45.312408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:48:45.312408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:45.312408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:45.312408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:45.312408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:45.313856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:45.313856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:45.313856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:45.313856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:45.313856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:45.313856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:45.313856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:45.313856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:45.313856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:45.313856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:45.313856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 22:48:45.313856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:48:45.313856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 146/305] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:45.313856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8453c08-3cb5-4b20-9e49-6c536e78fc47 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:45.378386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ecd65-99ff-41bf-b86b-e131e73955c9 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:45.410330 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8453c08-3cb5-4b20-9e49-6c536e78fc47 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:45.412364 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84a778-4276-4a0a-a849-fd5b3205994b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:45.413752 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84a778-4276-4a0a-a849-fd5b3205994b None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:45.414026 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84a778-4276-4a0a-a849-fd5b3205994b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:45.414476 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84a778-4276-4a0a-a849-fd5b3205994b None admin] PATH_INFO: `/v3/system/users/b91f416e19a5463e8b18d6e052634c41/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:45.415969 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84a778-4276-4a0a-a849-fd5b3205994b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b91f416e19a5463e8b18d6e052634c41, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:45.417335 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84a778-4276-4a0a-a849-fd5b3205994b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:45.426970 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 160/306] 10.222.0.23 () {58 vars in 1334 bytes} [Mon May 9 22:48:45 2022] PUT /identity/v3/system/users/b91f416e19a5463e8b18d6e052634c41/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:48:45.441178 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84a778-4276-4a0a-a849-fd5b3205994b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:45.460146 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8faa0a-ca3d-4ca5-a457-e0534a78e723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:45.460945 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8faa0a-ca3d-4ca5-a457-e0534a78e723 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:45.461301 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8faa0a-ca3d-4ca5-a457-e0534a78e723 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:45.461648 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8faa0a-ca3d-4ca5-a457-e0534a78e723 None admin] PATH_INFO: `/v3/groups` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:45.462580 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8faa0a-ca3d-4ca5-a457-e0534a78e723 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:45.464162 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8faa0a-ca3d-4ca5-a457-e0534a78e723 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:45.487436 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 147/307] 10.222.0.23 () {60 vars in 1134 bytes} [Mon May 9 22:48:45 2022] POST /identity/v3/groups => generated 229 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:48:45.572103 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84a778-4276-4a0a-a849-fd5b3205994b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:45.613316 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecab912-8eae-46b1-94d0-833594191b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:45.614437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecab912-8eae-46b1-94d0-833594191b2e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:45.614904 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecab912-8eae-46b1-94d0-833594191b2e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:45.615323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecab912-8eae-46b1-94d0-833594191b2e None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:45.616881 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecab912-8eae-46b1-94d0-833594191b2e None admin] RBAC: Adding query filter params (name=reader) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:45.617368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecab912-8eae-46b1-94d0-833594191b2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:45.619031 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecab912-8eae-46b1-94d0-833594191b2e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:45.629067 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 161/308] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:45.635974 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8faa0a-ca3d-4ca5-a457-e0534a78e723 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:45.746757 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8faa0a-ca3d-4ca5-a457-e0534a78e723 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:45.785148 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca9a1f-984f-40c4-9f68-99d592f59718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:45.786026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca9a1f-984f-40c4-9f68-99d592f59718 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:45.786327 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca9a1f-984f-40c4-9f68-99d592f59718 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:45.786603 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca9a1f-984f-40c4-9f68-99d592f59718 None admin] PATH_INFO: `/v3/users/7a2a0b3c138448d093d12616eab013d6` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:45.801201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca9a1f-984f-40c4-9f68-99d592f59718 None admin] RBAC: Authorizing `identity:get_user(user_id=7a2a0b3c138448d093d12616eab013d6)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:45.802710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca9a1f-984f-40c4-9f68-99d592f59718 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:45.804470 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 148/309] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:45 2022] GET /identity/v3/users/7a2a0b3c138448d093d12616eab013d6 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:45.810342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecab912-8eae-46b1-94d0-833594191b2e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:45.910350 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecab912-8eae-46b1-94d0-833594191b2e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:45.933196 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c7c16-9b0a-403e-a686-e01c0e5dfd67 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:45.933510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c7c16-9b0a-403e-a686-e01c0e5dfd67 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:45.933762 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c7c16-9b0a-403e-a686-e01c0e5dfd67 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:45.935422 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 149/310] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:48:45.940402 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ab747-1e00-45fc-9d83-d122651143d1 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:45.940691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ab747-1e00-45fc-9d83-d122651143d1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:45.940973 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ab747-1e00-45fc-9d83-d122651143d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:45.944640 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7c004-6569-4aff-8ebd-c136fa96983a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:45.945394 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7c004-6569-4aff-8ebd-c136fa96983a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:45.945676 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7c004-6569-4aff-8ebd-c136fa96983a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:45.945956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7c004-6569-4aff-8ebd-c136fa96983a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:45.946805 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb7c004-6569-4aff-8ebd-c136fa96983a None admin] RBAC: Adding query filter params (role.id=949d4917fcdb41e2a938aadb277d1970, user.id=7a2a0b3c138448d093d12616eab013d6, scope.system=all) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:45.947119 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb7c004-6569-4aff-8ebd-c136fa96983a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:45.948563 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb7c004-6569-4aff-8ebd-c136fa96983a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:45.953729 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 162/311] 10.222.0.23 () {56 vars in 1308 bytes} [Mon May 9 22:48:45 2022] GET /identity/v3/role_assignments?user.id=7a2a0b3c138448d093d12616eab013d6&scope.system=all&role.id=949d4917fcdb41e2a938aadb277d1970 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:45.959246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-452ab747-1e00-45fc-9d83-d122651143d1 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:45.961057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-452ab747-1e00-45fc-9d83-d122651143d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:46.038330 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 150/312] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:48:46.062340 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7c004-6569-4aff-8ebd-c136fa96983a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:46.169353 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7c004-6569-4aff-8ebd-c136fa96983a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:46.204108 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd882aa-fef8-41b3-8ab1-25c783031cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:46.205712 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd882aa-fef8-41b3-8ab1-25c783031cf5 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:46.206364 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd882aa-fef8-41b3-8ab1-25c783031cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:46.206883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd882aa-fef8-41b3-8ab1-25c783031cf5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:46.218847 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd882aa-fef8-41b3-8ab1-25c783031cf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:46.226917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-3fd882aa-fef8-41b3-8ab1-25c783031cf5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 22:48:46.226917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:46.226917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:46.226917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:46.226917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:46.226917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:46.226917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:46.226917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:46.226917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:46.226917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:46.226917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:46.226917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:46.226917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:46.226917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:46.226917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:46.226917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:46.227832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:46.227832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:46.227832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:46.227832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:46.227832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:46.227832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:46.227832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:46.227832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:46.227832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:46.227832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:46.227832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:46.227832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:46.227832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:46.227832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:46.227832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:46.227832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:46.228667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:46.228667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:46.228667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:46.228667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:46.228667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:46.228667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:46.228667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:46.228667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:46.228667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:46.228667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:46.228667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 22:48:46.228667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:46.231440 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 163/313] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:46.242151 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ab747-1e00-45fc-9d83-d122651143d1 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:46.338808 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ab747-1e00-45fc-9d83-d122651143d1 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:46.373129 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31db84f-1786-4ca8-b9a9-f45f18d90707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:46.373808 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31db84f-1786-4ca8-b9a9-f45f18d90707 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:46.374090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31db84f-1786-4ca8-b9a9-f45f18d90707 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:46.374395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31db84f-1786-4ca8-b9a9-f45f18d90707 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:46.375149 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31db84f-1786-4ca8-b9a9-f45f18d90707 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:46.375458 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31db84f-1786-4ca8-b9a9-f45f18d90707 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:46.376854 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31db84f-1786-4ca8-b9a9-f45f18d90707 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:46.386098 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 151/314] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:46.391609 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd882aa-fef8-41b3-8ab1-25c783031cf5 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:46.494230 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd882aa-fef8-41b3-8ab1-25c783031cf5 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:46.527582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97b197-8c24-4da9-9e9f-437814531841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:46.528446 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97b197-8c24-4da9-9e9f-437814531841 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:46.528722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97b197-8c24-4da9-9e9f-437814531841 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:46.528992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97b197-8c24-4da9-9e9f-437814531841 None admin] PATH_INFO: `/v3/users/a8b11af91c5045b3bc9ffbed69882fbc` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:46.530222 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97b197-8c24-4da9-9e9f-437814531841 None admin] RBAC: Authorizing `identity:get_user(user_id=a8b11af91c5045b3bc9ffbed69882fbc)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:46.531659 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97b197-8c24-4da9-9e9f-437814531841 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:46.533620 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 164/315] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:46 2022] GET /identity/v3/users/a8b11af91c5045b3bc9ffbed69882fbc => generated 297 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:46.538706 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31db84f-1786-4ca8-b9a9-f45f18d90707 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:46.650727 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31db84f-1786-4ca8-b9a9-f45f18d90707 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:46.685307 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab735ee-05e8-4194-8a0f-1eee3611a13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:46.685961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab735ee-05e8-4194-8a0f-1eee3611a13d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:46.686274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab735ee-05e8-4194-8a0f-1eee3611a13d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:46.686550 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab735ee-05e8-4194-8a0f-1eee3611a13d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:46.758942 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab735ee-05e8-4194-8a0f-1eee3611a13d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:46.760300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab735ee-05e8-4194-8a0f-1eee3611a13d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:46.860900 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 152/316] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:48:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:48:46.866320 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97b197-8c24-4da9-9e9f-437814531841 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:46.965266 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97b197-8c24-4da9-9e9f-437814531841 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:46.998548 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425779f-d4dd-4227-8680-77fde9bf6eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:46.999397 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425779f-d4dd-4227-8680-77fde9bf6eda None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:46.999680 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425779f-d4dd-4227-8680-77fde9bf6eda None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:46.999952 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425779f-d4dd-4227-8680-77fde9bf6eda None admin] PATH_INFO: `/v3/projects/db7334eab91d41fa884c491d1364a253` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:47.001127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0425779f-d4dd-4227-8680-77fde9bf6eda None admin] RBAC: Authorizing `identity:get_project(project_id=db7334eab91d41fa884c491d1364a253)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:47.002589 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0425779f-d4dd-4227-8680-77fde9bf6eda None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:47.004271 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 165/317] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:46 2022] GET /identity/v3/projects/db7334eab91d41fa884c491d1364a253 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:47.010767 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab735ee-05e8-4194-8a0f-1eee3611a13d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:47.109290 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab735ee-05e8-4194-8a0f-1eee3611a13d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:47.146969 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624072fe-6557-4308-a88d-2fc16c01172b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:47.148168 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624072fe-6557-4308-a88d-2fc16c01172b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:47.148448 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624072fe-6557-4308-a88d-2fc16c01172b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:47.148721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624072fe-6557-4308-a88d-2fc16c01172b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:47.149606 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624072fe-6557-4308-a88d-2fc16c01172b None admin] RBAC: Adding query filter params (role.id=949d4917fcdb41e2a938aadb277d1970, scope.project.id=db7334eab91d41fa884c491d1364a253, user.id=a8b11af91c5045b3bc9ffbed69882fbc) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:47.149921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624072fe-6557-4308-a88d-2fc16c01172b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:47.151436 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624072fe-6557-4308-a88d-2fc16c01172b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:47.157735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 153/318] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:48:47 2022] GET /identity/v3/role_assignments?user.id=a8b11af91c5045b3bc9ffbed69882fbc&scope.project.id=db7334eab91d41fa884c491d1364a253&role.id=949d4917fcdb41e2a938aadb277d1970 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:47.736763 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3887633-532a-4bca-8943-8d89d897f06a None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:47.737127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3887633-532a-4bca-8943-8d89d897f06a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:47.737323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3887633-532a-4bca-8943-8d89d897f06a None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:47.738871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 166/319] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:48:47.744418 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f1a03-f0ca-4df4-8de0-fff0bebfba8b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:47.744846 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f1a03-f0ca-4df4-8de0-fff0bebfba8b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:47.745004 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f1a03-f0ca-4df4-8de0-fff0bebfba8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:47.761728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e06f1a03-f0ca-4df4-8de0-fff0bebfba8b None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:47.762934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e06f1a03-f0ca-4df4-8de0-fff0bebfba8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:47.828555 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 154/320] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:47.850948 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3887633-532a-4bca-8943-8d89d897f06a None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:47.867428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c202297-a0ac-41e7-8206-6836c2525c91 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:47.867707 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c202297-a0ac-41e7-8206-6836c2525c91 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:47.867992 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c202297-a0ac-41e7-8206-6836c2525c91 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:47.869344 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 155/321] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:48:47.874177 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415c235-6f38-4d13-bd73-3a0e9696bd2d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:47.874468 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415c235-6f38-4d13-bd73-3a0e9696bd2d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:47.874743 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415c235-6f38-4d13-bd73-3a0e9696bd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:47.891661 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c415c235-6f38-4d13-bd73-3a0e9696bd2d None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:47.892760 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c415c235-6f38-4d13-bd73-3a0e9696bd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:47.951891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3887633-532a-4bca-8943-8d89d897f06a None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:47.959016 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 156/322] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:48:47.980882 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415c235-6f38-4d13-bd73-3a0e9696bd2d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:47.986699 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951430c6-c758-46c6-91c8-1466afd3eb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:47.987489 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951430c6-c758-46c6-91c8-1466afd3eb7b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:47.987762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951430c6-c758-46c6-91c8-1466afd3eb7b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:47.988027 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951430c6-c758-46c6-91c8-1466afd3eb7b None admin] PATH_INFO: `/v3/roles/member` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:47.998479 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951430c6-c758-46c6-91c8-1466afd3eb7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:48.005973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-951430c6-c758-46c6-91c8-1466afd3eb7b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:48.005973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:48.005973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:48.005973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:48.005973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:48.005973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:48.005973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:48.005973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:48.005973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:48.005973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:48.005973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:48.005973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:48.005973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:48.005973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:48.005973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:48.005973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:48.007582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:48.007582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:48.007582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:48.007582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:48.007582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:48.007582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:48.007582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:48.007582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:48.007582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:48.007582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:48.007582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:48.007582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:48.007582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:48.007582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:48.007582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:48.007582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:48.008392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:48.008392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:48.008392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:48.008392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:48.008392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:48.008392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:48.008392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:48.008392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:48.008392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:48.008392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:48.008392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:48.008392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:48.008392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 167/323] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:48.013253 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951430c6-c758-46c6-91c8-1466afd3eb7b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:48.082447 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415c235-6f38-4d13-bd73-3a0e9696bd2d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:48.114512 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951430c6-c758-46c6-91c8-1466afd3eb7b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:48.117478 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6558d702-cd12-414a-a976-00685b24490a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:48.118291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558d702-cd12-414a-a976-00685b24490a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:48.118575 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558d702-cd12-414a-a976-00685b24490a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:48.118845 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558d702-cd12-414a-a976-00685b24490a None admin] PATH_INFO: `/v3/domains/default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:48.119999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6558d702-cd12-414a-a976-00685b24490a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:48.121450 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6558d702-cd12-414a-a976-00685b24490a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:48.123084 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 157/324] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:48:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:48.128458 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6558d702-cd12-414a-a976-00685b24490a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:48.149342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c912dfd-f6af-4940-9fd5-9cc3625205e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:48.149941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c912dfd-f6af-4940-9fd5-9cc3625205e2 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:48.150243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c912dfd-f6af-4940-9fd5-9cc3625205e2 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:48.150515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c912dfd-f6af-4940-9fd5-9cc3625205e2 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:48.151252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c912dfd-f6af-4940-9fd5-9cc3625205e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:48.151554 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c912dfd-f6af-4940-9fd5-9cc3625205e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:48.152912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c912dfd-f6af-4940-9fd5-9cc3625205e2 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:48.161075 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 168/325] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:48.166258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c912dfd-f6af-4940-9fd5-9cc3625205e2 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:48.229500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6558d702-cd12-414a-a976-00685b24490a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:48.261477 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c912dfd-f6af-4940-9fd5-9cc3625205e2 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:48.262869 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf397fb4-8b2a-4447-87be-b19ba7f1c729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:48.263525 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf397fb4-8b2a-4447-87be-b19ba7f1c729 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:48.263803 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf397fb4-8b2a-4447-87be-b19ba7f1c729 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:48.264072 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf397fb4-8b2a-4447-87be-b19ba7f1c729 None admin] PATH_INFO: `/v3/groups` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:48.264742 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf397fb4-8b2a-4447-87be-b19ba7f1c729 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:48.266203 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf397fb4-8b2a-4447-87be-b19ba7f1c729 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:48.279713 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 158/326] 10.222.0.23 () {60 vars in 1134 bytes} [Mon May 9 22:48:48 2022] POST /identity/v3/groups => generated 226 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 22:48:48.280951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077cb0a-e265-4eb1-9f43-323389a555f2 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:48.281236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077cb0a-e265-4eb1-9f43-323389a555f2 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:48.281504 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077cb0a-e265-4eb1-9f43-323389a555f2 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:48.283248 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 159/327] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:48.288989 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cb996-2b78-485b-91e7-ec951584f09b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:48.289273 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cb996-2b78-485b-91e7-ec951584f09b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:48.289543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cb996-2b78-485b-91e7-ec951584f09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:48.296463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b744d-506f-432b-a23c-32ffb5c404d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:48.297375 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b744d-506f-432b-a23c-32ffb5c404d4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:48.297655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b744d-506f-432b-a23c-32ffb5c404d4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:48.297918 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b744d-506f-432b-a23c-32ffb5c404d4 None admin] PATH_INFO: `/v3/users/b91f416e19a5463e8b18d6e052634c41` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:48.299237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b744d-506f-432b-a23c-32ffb5c404d4 None admin] RBAC: Authorizing `identity:get_user(user_id=b91f416e19a5463e8b18d6e052634c41)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:48.300761 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b744d-506f-432b-a23c-32ffb5c404d4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:48.302547 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 169/328] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:48 2022] GET /identity/v3/users/b91f416e19a5463e8b18d6e052634c41 => generated 293 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:48.307541 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-cf2cb996-2b78-485b-91e7-ec951584f09b None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:48.308756 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b744d-506f-432b-a23c-32ffb5c404d4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:48.308877 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-cf2cb996-2b78-485b-91e7-ec951584f09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:48.390537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 160/329] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:48:48.414529 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cb996-2b78-485b-91e7-ec951584f09b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:48.416740 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b744d-506f-432b-a23c-32ffb5c404d4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:48.451333 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164da314-1586-4da3-a6c2-787001844dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:48.451993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164da314-1586-4da3-a6c2-787001844dd6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:48.452269 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164da314-1586-4da3-a6c2-787001844dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:48.452538 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164da314-1586-4da3-a6c2-787001844dd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:48.453359 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164da314-1586-4da3-a6c2-787001844dd6 None admin] RBAC: Adding query filter params (role.id=3631750e6bf34730aa3557cd90aa6108, user.id=b91f416e19a5463e8b18d6e052634c41, scope.system=all) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:48.453665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164da314-1586-4da3-a6c2-787001844dd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:48.455134 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164da314-1586-4da3-a6c2-787001844dd6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:48.469364 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 170/330] 10.222.0.23 () {56 vars in 1308 bytes} [Mon May 9 22:48:48 2022] GET /identity/v3/role_assignments?user.id=b91f416e19a5463e8b18d6e052634c41&scope.system=all&role.id=3631750e6bf34730aa3557cd90aa6108 => generated 510 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:48.528571 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cb996-2b78-485b-91e7-ec951584f09b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:48.567729 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704639e7-3f9c-4496-b622-8acb59897234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:48.568535 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704639e7-3f9c-4496-b622-8acb59897234 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:48.568820 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704639e7-3f9c-4496-b622-8acb59897234 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:48.569669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704639e7-3f9c-4496-b622-8acb59897234 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:48.581922 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704639e7-3f9c-4496-b622-8acb59897234 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:48.590624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-704639e7-3f9c-4496-b622-8acb59897234 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 22:48:48.590624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:48.590624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:48.590624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:48.590624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:48.590624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:48.590624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:48.590624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:48.590624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:48.590624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:48.590624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:48.590624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:48.590624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:48.590624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:48.590624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:48.590624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:48.591731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:48.591731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:48.591731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:48.591731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:48.591731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:48.591731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:48.591731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:48.591731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:48.591731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:48.591731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:48.591731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:48.591731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:48:48.591731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:48.591731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:48.591731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:48.591731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:48.592720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:48.592720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:48.592720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:48.592720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:48.592720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:48.592720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:48.592720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:48.592720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:48.592720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:48.592720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:48.592720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 22:48:48.592720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:48:48.592720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 161/331] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:48.598834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164da314-1586-4da3-a6c2-787001844dd6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:48.716496 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164da314-1586-4da3-a6c2-787001844dd6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:48.753595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0081ae0-bd22-48dd-851a-51d22c72caac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:48.754306 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0081ae0-bd22-48dd-851a-51d22c72caac None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:48.754586 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0081ae0-bd22-48dd-851a-51d22c72caac None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:48.754862 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0081ae0-bd22-48dd-851a-51d22c72caac None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:48.755611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0081ae0-bd22-48dd-851a-51d22c72caac None admin] RBAC: Adding query filter params (name=reader) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:48.755914 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0081ae0-bd22-48dd-851a-51d22c72caac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:48.757371 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0081ae0-bd22-48dd-851a-51d22c72caac None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:48.766216 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 171/332] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:48.772774 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704639e7-3f9c-4496-b622-8acb59897234 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:48.875650 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704639e7-3f9c-4496-b622-8acb59897234 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:48.911284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b01eb-c3f8-4acc-87ec-306f77040c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:48.912495 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b01eb-c3f8-4acc-87ec-306f77040c95 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:48.913046 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b01eb-c3f8-4acc-87ec-306f77040c95 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:48.913555 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b01eb-c3f8-4acc-87ec-306f77040c95 None admin] PATH_INFO: `/v3/users/7a2a0b3c138448d093d12616eab013d6` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:48.915131 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73b01eb-c3f8-4acc-87ec-306f77040c95 None admin] RBAC: Authorizing `identity:get_user(user_id=7a2a0b3c138448d093d12616eab013d6)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:48.916796 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73b01eb-c3f8-4acc-87ec-306f77040c95 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:48.918788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 162/333] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:48 2022] GET /identity/v3/users/7a2a0b3c138448d093d12616eab013d6 => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:48.924444 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0081ae0-bd22-48dd-851a-51d22c72caac None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:49.032627 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0081ae0-bd22-48dd-851a-51d22c72caac None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:49.067654 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4bb39a-4119-43f5-bb4e-6e8f74984707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:49.069413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bb39a-4119-43f5-bb4e-6e8f74984707 None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:49.069913 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bb39a-4119-43f5-bb4e-6e8f74984707 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:49.070407 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bb39a-4119-43f5-bb4e-6e8f74984707 None admin] PATH_INFO: `/v3/system/users/7a2a0b3c138448d093d12616eab013d6/roles/949d4917fcdb41e2a938aadb277d1970` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:49.072206 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4bb39a-4119-43f5-bb4e-6e8f74984707 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7a2a0b3c138448d093d12616eab013d6, role_id=949d4917fcdb41e2a938aadb277d1970)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:49.073824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4bb39a-4119-43f5-bb4e-6e8f74984707 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:49.081821 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 172/334] 10.222.0.23 () {58 vars in 1334 bytes} [Mon May 9 22:48:48 2022] PUT /identity/v3/system/users/7a2a0b3c138448d093d12616eab013d6/roles/949d4917fcdb41e2a938aadb277d1970 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:48:50.830301 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8dd97-78ce-42b6-a05b-bb8b4793caf9 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:50.831289 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8dd97-78ce-42b6-a05b-bb8b4793caf9 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:50.831770 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8dd97-78ce-42b6-a05b-bb8b4793caf9 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:50.833575 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 163/335] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:50.838741 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48524a0a-643a-46fb-b655-f204070ad6dd None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:50.839561 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48524a0a-643a-46fb-b655-f204070ad6dd None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:50.840045 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48524a0a-643a-46fb-b655-f204070ad6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:50.863124 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-48524a0a-643a-46fb-b655-f204070ad6dd None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:50.864616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-48524a0a-643a-46fb-b655-f204070ad6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:50.943081 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 173/336] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:50.983641 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8dd97-78ce-42b6-a05b-bb8b4793caf9 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:51.103905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8dd97-78ce-42b6-a05b-bb8b4793caf9 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:51.145647 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25aca96-f3da-4339-92d8-00de7d8c452f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:51.146479 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25aca96-f3da-4339-92d8-00de7d8c452f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:51.146758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25aca96-f3da-4339-92d8-00de7d8c452f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:51.147064 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25aca96-f3da-4339-92d8-00de7d8c452f None admin] PATH_INFO: `/v3/roles/member` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:51.159857 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25aca96-f3da-4339-92d8-00de7d8c452f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:51.168326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-d25aca96-f3da-4339-92d8-00de7d8c452f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:51.168326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:51.168326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:51.168326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:51.168326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:51.168326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:51.168326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:51.168326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:51.168326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:51.168326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:51.168326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:51.168326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:51.168326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:51.168326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:51.168326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:51.168326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:51.169208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:51.169208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:51.169208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:51.169208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:51.169208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:51.169208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:51.169208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:51.169208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:51.169208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:51.169208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:51.169208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:51.169208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:48:51.169208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:51.169208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:51.169208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:51.169208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:51.170000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:51.170000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:51.170000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:51.170000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:51.170000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:51.170000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:51.170000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:51.170000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:51.170000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:51.170000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:51.170000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:51.170000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:48:51.170000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 164/337] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:51.175484 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48524a0a-643a-46fb-b655-f204070ad6dd None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:51.295730 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48524a0a-643a-46fb-b655-f204070ad6dd None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:51.339032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b1185-f7a8-4f88-837c-5642bf51672d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:51.339756 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b1185-f7a8-4f88-837c-5642bf51672d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:51.340041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b1185-f7a8-4f88-837c-5642bf51672d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:51.340317 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b1185-f7a8-4f88-837c-5642bf51672d None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:51.341124 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b1185-f7a8-4f88-837c-5642bf51672d None admin] RBAC: Adding query filter params (name=member) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:51.341430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b1185-f7a8-4f88-837c-5642bf51672d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:51.342925 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b1185-f7a8-4f88-837c-5642bf51672d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:51.359336 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 174/338] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:51.365359 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25aca96-f3da-4339-92d8-00de7d8c452f None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:51.537472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25aca96-f3da-4339-92d8-00de7d8c452f None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:51.579921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5a94c-f3e8-4d0e-8a16-483c7d5403d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:51.580499 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5a94c-f3e8-4d0e-8a16-483c7d5403d8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:51.580783 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5a94c-f3e8-4d0e-8a16-483c7d5403d8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:51.581059 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5a94c-f3e8-4d0e-8a16-483c7d5403d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:51.664760 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5a94c-f3e8-4d0e-8a16-483c7d5403d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:51.665427 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c850e8-55ae-48d7-8ff7-737d049816ee None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:51.665952 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c850e8-55ae-48d7-8ff7-737d049816ee None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:51.666487 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c850e8-55ae-48d7-8ff7-737d049816ee None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:51.666592 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5a94c-f3e8-4d0e-8a16-483c7d5403d8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:51.668191 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 175/339] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:48:51.674679 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6d1b6-8f02-4f7d-a268-516fbb89b9b6 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:51.675242 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6d1b6-8f02-4f7d-a268-516fbb89b9b6 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:51.675722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6d1b6-8f02-4f7d-a268-516fbb89b9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:51.705251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-85c6d1b6-8f02-4f7d-a268-516fbb89b9b6 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:51.710479 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-85c6d1b6-8f02-4f7d-a268-516fbb89b9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:51.799892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 176/340] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:51.802774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3e8a4-3422-469c-a3c7-6ebbaa446816 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:51.803410 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3e8a4-3422-469c-a3c7-6ebbaa446816 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:51.803695 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3e8a4-3422-469c-a3c7-6ebbaa446816 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:51.805290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 177/341] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:51.812342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7281bd-bcc8-4f45-b00e-e4cdba75282a None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:51.812631 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7281bd-bcc8-4f45-b00e-e4cdba75282a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:51.812908 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7281bd-bcc8-4f45-b00e-e4cdba75282a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:51.834265 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 165/342] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:48:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:48:51.843144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5a94c-f3e8-4d0e-8a16-483c7d5403d8 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:51.855142 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ce7281bd-bcc8-4f45-b00e-e4cdba75282a None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:51.856345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ce7281bd-bcc8-4f45-b00e-e4cdba75282a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:51.944751 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5a94c-f3e8-4d0e-8a16-483c7d5403d8 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:51.979594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152da2d-f927-4706-82c0-02b650ce7c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:51.980459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152da2d-f927-4706-82c0-02b650ce7c75 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:51.980738 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152da2d-f927-4706-82c0-02b650ce7c75 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:51.981011 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152da2d-f927-4706-82c0-02b650ce7c75 None admin] PATH_INFO: `/v3/projects/1a6770d749f644eea4fd1584b1503d28` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:51.982319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152da2d-f927-4706-82c0-02b650ce7c75 None admin] RBAC: Authorizing `identity:get_project(project_id=1a6770d749f644eea4fd1584b1503d28)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:51.983776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152da2d-f927-4706-82c0-02b650ce7c75 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:51.986049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 166/343] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:51 2022] GET /identity/v3/projects/1a6770d749f644eea4fd1584b1503d28 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:51.986902 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152da2d-f927-4706-82c0-02b650ce7c75 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:52.017364 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 178/344] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:52.018266 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7281bd-bcc8-4f45-b00e-e4cdba75282a None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:52.099491 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152da2d-f927-4706-82c0-02b650ce7c75 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:52.134283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5d4bc-5175-4840-a167-c8359f328729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:52.135095 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5d4bc-5175-4840-a167-c8359f328729 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:52.135424 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5d4bc-5175-4840-a167-c8359f328729 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:52.135705 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5d4bc-5175-4840-a167-c8359f328729 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:52.146273 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5d4bc-5175-4840-a167-c8359f328729 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:52.155346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-e9f5d4bc-5175-4840-a167-c8359f328729 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:48:52.155346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:52.155346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:52.155346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:52.155346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:52.155346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:52.155346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:52.155346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:52.155346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:52.155346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:52.155346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:52.155346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:52.155346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:52.155346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:52.155346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:52.155346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:52.157473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:52.157473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:52.157473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:52.157473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:52.157473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:52.157473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:52.157473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:52.157473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:52.157473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:52.157473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:52.157473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:52.157473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:48:52.157473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:52.157473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:52.157473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:52.157473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:52.159464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:52.159464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:52.159464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:52.159464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:52.159464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:52.159464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:52.159464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:52.159464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:52.159464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:52.159464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:52.159464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:48:52.159464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:48:52.159464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 167/345] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 9 22:48:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:52.159464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5d4bc-5175-4840-a167-c8359f328729 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:52.236587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7281bd-bcc8-4f45-b00e-e4cdba75282a None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:52.264266 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5d4bc-5175-4840-a167-c8359f328729 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:52.274394 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e99ee6-f8ae-4491-a1e4-b1f4a59e8df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:52.275265 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e99ee6-f8ae-4491-a1e4-b1f4a59e8df7 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:52.275544 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e99ee6-f8ae-4491-a1e4-b1f4a59e8df7 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:52.275808 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e99ee6-f8ae-4491-a1e4-b1f4a59e8df7 None admin] PATH_INFO: `/v3/groups/d2102ce0e44a47a58cffd1e370f61bdd` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:52.282480 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e99ee6-f8ae-4491-a1e4-b1f4a59e8df7 None admin] RBAC: Authorizing `identity:get_group(group_id=d2102ce0e44a47a58cffd1e370f61bdd)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:52.284377 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e99ee6-f8ae-4491-a1e4-b1f4a59e8df7 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:52.285963 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 179/346] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 9 22:48:52 2022] GET /identity/v3/groups/d2102ce0e44a47a58cffd1e370f61bdd => generated 226 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:52.286762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e99ee6-f8ae-4491-a1e4-b1f4a59e8df7 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:52.305842 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc498a-d888-4bd1-983b-60b5eb8cd0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:52.306694 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc498a-d888-4bd1-983b-60b5eb8cd0fd None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:52.306975 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc498a-d888-4bd1-983b-60b5eb8cd0fd None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:52.307249 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc498a-d888-4bd1-983b-60b5eb8cd0fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:52.319820 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccc498a-d888-4bd1-983b-60b5eb8cd0fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:52.327847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-6ccc498a-d888-4bd1-983b-60b5eb8cd0fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:52.327847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:52.327847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:52.327847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:52.327847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:52.327847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:52.327847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:52.327847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:52.327847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:52.327847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:52.327847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:52.327847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:52.327847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:52.327847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:52.327847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:52.327847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:52.329979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:52.329979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:52.329979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:52.329979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:52.329979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:52.329979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:52.329979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:52.329979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:52.329979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:52.329979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:52.329979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:52.329979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:48:52.329979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:52.329979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:52.329979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:52.329979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:52.331999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:52.331999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:52.331999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:52.331999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:52.331999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:52.331999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:52.331999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:52.331999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:52.331999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:52.331999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:52.331999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:52.331999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:48:52.331999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 168/347] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:52.331999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc498a-d888-4bd1-983b-60b5eb8cd0fd None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:52.408496 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e99ee6-f8ae-4491-a1e4-b1f4a59e8df7 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:52.447487 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e205b73-4145-42d3-b334-345c07b9e63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:52.448022 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc498a-d888-4bd1-983b-60b5eb8cd0fd None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:52.448223 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e205b73-4145-42d3-b334-345c07b9e63c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:52.448413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e205b73-4145-42d3-b334-345c07b9e63c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:52.448684 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e205b73-4145-42d3-b334-345c07b9e63c None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:52.449426 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e205b73-4145-42d3-b334-345c07b9e63c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:52.449727 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e205b73-4145-42d3-b334-345c07b9e63c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:52.451135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e205b73-4145-42d3-b334-345c07b9e63c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:52.459602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 180/348] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:48:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:52.460229 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e205b73-4145-42d3-b334-345c07b9e63c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:52.482690 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b933a15-faa1-4341-ae6d-fc89b4403de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:52.483404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b933a15-faa1-4341-ae6d-fc89b4403de3 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:52.483688 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b933a15-faa1-4341-ae6d-fc89b4403de3 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:52.484021 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b933a15-faa1-4341-ae6d-fc89b4403de3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:52.485126 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b933a15-faa1-4341-ae6d-fc89b4403de3 None admin] RBAC: Adding query filter params (group.id=d2102ce0e44a47a58cffd1e370f61bdd, role.id=3631750e6bf34730aa3557cd90aa6108, scope.project.id=1a6770d749f644eea4fd1584b1503d28) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:52.485442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b933a15-faa1-4341-ae6d-fc89b4403de3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:52.486977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b933a15-faa1-4341-ae6d-fc89b4403de3 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:52.493560 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 169/349] 10.222.0.23 () {56 vars in 1376 bytes} [Mon May 9 22:48:52 2022] GET /identity/v3/role_assignments?group.id=d2102ce0e44a47a58cffd1e370f61bdd&scope.project.id=1a6770d749f644eea4fd1584b1503d28&role.id=3631750e6bf34730aa3557cd90aa6108 => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:52.499218 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b933a15-faa1-4341-ae6d-fc89b4403de3 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:52.563932 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e205b73-4145-42d3-b334-345c07b9e63c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:52.599697 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c946ddb-4daa-4e6c-be8e-0047df509675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:52.600899 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c946ddb-4daa-4e6c-be8e-0047df509675 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:52.601174 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c946ddb-4daa-4e6c-be8e-0047df509675 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:52.601441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c946ddb-4daa-4e6c-be8e-0047df509675 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:52.602234 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c946ddb-4daa-4e6c-be8e-0047df509675 None admin] RBAC: Adding query filter params (name=member) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:52.602539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c946ddb-4daa-4e6c-be8e-0047df509675 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:52.603904 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c946ddb-4daa-4e6c-be8e-0047df509675 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:52.613233 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 181/350] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:52.614719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b01bf-d67d-42a5-8961-18ea4cad40e4 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:52.614923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b01bf-d67d-42a5-8961-18ea4cad40e4 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:52.615164 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b01bf-d67d-42a5-8961-18ea4cad40e4 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:52.617167 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 182/351] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:52.627609 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae24bf-4806-4f47-bbdb-6d94672ca126 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:52.627894 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae24bf-4806-4f47-bbdb-6d94672ca126 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:52.628168 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae24bf-4806-4f47-bbdb-6d94672ca126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:52.637450 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b933a15-faa1-4341-ae6d-fc89b4403de3 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:52.652680 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a9ae24bf-4806-4f47-bbdb-6d94672ca126 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:52.653910 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a9ae24bf-4806-4f47-bbdb-6d94672ca126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:52.677503 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01990c0-fc40-40a7-82e0-02d1e3078050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:52.678080 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01990c0-fc40-40a7-82e0-02d1e3078050 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:52.678962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01990c0-fc40-40a7-82e0-02d1e3078050 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:52.679240 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01990c0-fc40-40a7-82e0-02d1e3078050 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:52.727964 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 183/352] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:52.728845 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae24bf-4806-4f47-bbdb-6d94672ca126 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:52.764132 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01990c0-fc40-40a7-82e0-02d1e3078050 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:52.766650 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01990c0-fc40-40a7-82e0-02d1e3078050 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:52.834191 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae24bf-4806-4f47-bbdb-6d94672ca126 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:52.880445 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79ffb0-c506-4384-8603-4691f2415fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:52.881902 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79ffb0-c506-4384-8603-4691f2415fea None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:52.882386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79ffb0-c506-4384-8603-4691f2415fea None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:52.882807 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79ffb0-c506-4384-8603-4691f2415fea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:52.884217 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 170/353] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:48:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:48:52.890197 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b8355-b11c-47c1-9ce6-d663d30e8979 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:52.890197 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b8355-b11c-47c1-9ce6-d663d30e8979 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:52.890197 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b8355-b11c-47c1-9ce6-d663d30e8979 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:52.890197 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 171/354] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:52.890197 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b8355-b11c-47c1-9ce6-d663d30e8979 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:52.975867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a79ffb0-c506-4384-8603-4691f2415fea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:52.977623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a79ffb0-c506-4384-8603-4691f2415fea None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:52.997476 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b8355-b11c-47c1-9ce6-d663d30e8979 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:53.042042 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a7bab-1593-4f13-8f8c-79c6bf231636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:53.043081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a7bab-1593-4f13-8f8c-79c6bf231636 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:53.043487 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a7bab-1593-4f13-8f8c-79c6bf231636 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:53.043873 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a7bab-1593-4f13-8f8c-79c6bf231636 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:53.056367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a7bab-1593-4f13-8f8c-79c6bf231636 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:53.067626 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-f25a7bab-1593-4f13-8f8c-79c6bf231636 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:48:53.067626 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:53.067626 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:53.067626 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:53.067626 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:53.067626 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:53.067626 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:53.067626 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:53.067626 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:53.067626 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:53.067626 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:53.067626 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:53.067626 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:53.067626 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:53.067626 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:53.067626 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:53.068591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:53.068591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:53.068591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:53.068591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:53.068591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:53.068591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:53.068591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:53.068591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:53.068591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:53.068591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:53.068591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:53.068591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:48:53.068591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:53.068591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:53.068591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:53.068591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:53.069432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:53.069432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:53.069432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:53.069432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:53.069432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:53.069432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:53.069432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:53.069432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:53.069432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:53.069432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:53.069432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:48:53.069432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:48:53.069432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 172/355] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 9 22:48:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:53.074288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a7bab-1593-4f13-8f8c-79c6bf231636 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:53.094979 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 184/356] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:48:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:48:53.096550 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894c098-b447-453c-8d8e-832f010fba95 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:53.097115 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894c098-b447-453c-8d8e-832f010fba95 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:53.097568 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894c098-b447-453c-8d8e-832f010fba95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:53.114895 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-8894c098-b447-453c-8d8e-832f010fba95 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:53.116375 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8894c098-b447-453c-8d8e-832f010fba95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:53.173913 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a7bab-1593-4f13-8f8c-79c6bf231636 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:53.184998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 185/357] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:53.186683 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894c098-b447-453c-8d8e-832f010fba95 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:53.208062 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a424e-a7f4-413a-b9e9-d45494544a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:53.208932 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a424e-a7f4-413a-b9e9-d45494544a10 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:53.209293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a424e-a7f4-413a-b9e9-d45494544a10 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:53.210217 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a424e-a7f4-413a-b9e9-d45494544a10 None admin] PATH_INFO: `/v3/projects/1a6770d749f644eea4fd1584b1503d28` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:53.211506 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a424e-a7f4-413a-b9e9-d45494544a10 None admin] RBAC: Authorizing `identity:get_project(project_id=1a6770d749f644eea4fd1584b1503d28)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:53.213054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a424e-a7f4-413a-b9e9-d45494544a10 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:53.215198 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 173/358] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:53 2022] GET /identity/v3/projects/1a6770d749f644eea4fd1584b1503d28 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:53.216035 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a424e-a7f4-413a-b9e9-d45494544a10 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:53.295143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894c098-b447-453c-8d8e-832f010fba95 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:53.317594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a424e-a7f4-413a-b9e9-d45494544a10 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:53.330147 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae27052-1e2b-4af1-9743-d903c8fa0a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:53.331135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae27052-1e2b-4af1-9743-d903c8fa0a6c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:53.331638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae27052-1e2b-4af1-9743-d903c8fa0a6c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:53.332127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae27052-1e2b-4af1-9743-d903c8fa0a6c None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:53.333069 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae27052-1e2b-4af1-9743-d903c8fa0a6c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:53.333576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae27052-1e2b-4af1-9743-d903c8fa0a6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:53.335718 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae27052-1e2b-4af1-9743-d903c8fa0a6c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:53.344688 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 186/359] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:48:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:53.345819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae27052-1e2b-4af1-9743-d903c8fa0a6c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:53.351994 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb87de-4537-4fe9-a0b5-e362f10bfd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:53.353429 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb87de-4537-4fe9-a0b5-e362f10bfd46 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:53.353429 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb87de-4537-4fe9-a0b5-e362f10bfd46 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:53.353429 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb87de-4537-4fe9-a0b5-e362f10bfd46 None admin] PATH_INFO: `/v3/projects/db7334eab91d41fa884c491d1364a253` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:53.354672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb87de-4537-4fe9-a0b5-e362f10bfd46 None admin] RBAC: Authorizing `identity:get_project(project_id=db7334eab91d41fa884c491d1364a253)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:53.356198 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb87de-4537-4fe9-a0b5-e362f10bfd46 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:53.358371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 174/360] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:53 2022] GET /identity/v3/projects/db7334eab91d41fa884c491d1364a253 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:53.359288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb87de-4537-4fe9-a0b5-e362f10bfd46 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:53.463883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae27052-1e2b-4af1-9743-d903c8fa0a6c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:53.472149 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb87de-4537-4fe9-a0b5-e362f10bfd46 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:53.499183 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70fa5b-3093-45f5-a9c4-4403f3561d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:53.500476 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70fa5b-3093-45f5-a9c4-4403f3561d99 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:53.500963 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70fa5b-3093-45f5-a9c4-4403f3561d99 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:53.501444 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70fa5b-3093-45f5-a9c4-4403f3561d99 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:53.505907 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813ba46-0c10-427a-b0a5-c9918843d04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:53.507386 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813ba46-0c10-427a-b0a5-c9918843d04b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:53.507665 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813ba46-0c10-427a-b0a5-c9918843d04b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:53.507941 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813ba46-0c10-427a-b0a5-c9918843d04b None admin] PATH_INFO: `/v3/groups/d2102ce0e44a47a58cffd1e370f61bdd` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:53.509165 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813ba46-0c10-427a-b0a5-c9918843d04b None admin] RBAC: Authorizing `identity:get_group(group_id=d2102ce0e44a47a58cffd1e370f61bdd)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:53.510619 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813ba46-0c10-427a-b0a5-c9918843d04b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:53.512225 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 175/361] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 9 22:48:53 2022] GET /identity/v3/groups/d2102ce0e44a47a58cffd1e370f61bdd => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:53.513065 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813ba46-0c10-427a-b0a5-c9918843d04b None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:53.515714 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70fa5b-3093-45f5-a9c4-4403f3561d99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:53.524806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-3e70fa5b-3093-45f5-a9c4-4403f3561d99 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 22:48:53.524806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:53.524806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:53.524806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:53.524806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:53.524806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:53.524806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:53.524806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:53.524806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:53.524806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:53.524806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:53.524806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:53.524806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:53.524806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:53.524806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:53.524806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:53.525731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:53.525731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:53.525731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:53.525731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:53.525731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:53.525731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:53.525731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:53.525731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:53.525731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:53.525731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:53.525731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:53.525731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:53.525731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:53.525731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:53.525731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:53.525731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:53.526572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:53.526572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:53.526572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:53.526572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:53.526572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:53.526572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:53.526572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:53.526572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:53.526572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:53.526572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:53.526572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 22:48:53.526572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:53.529233 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 187/362] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:53.530268 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70fa5b-3093-45f5-a9c4-4403f3561d99 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:53.617188 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813ba46-0c10-427a-b0a5-c9918843d04b None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:53.632406 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70fa5b-3093-45f5-a9c4-4403f3561d99 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:53.652901 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900e49b-1561-41e6-a779-4fdc833887ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:53.653755 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900e49b-1561-41e6-a779-4fdc833887ff None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:53.654028 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900e49b-1561-41e6-a779-4fdc833887ff None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:53.654327 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900e49b-1561-41e6-a779-4fdc833887ff None admin] PATH_INFO: `/v3/groups/d2102ce0e44a47a58cffd1e370f61bdd` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:53.655479 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900e49b-1561-41e6-a779-4fdc833887ff None admin] RBAC: Authorizing `identity:get_group(group_id=d2102ce0e44a47a58cffd1e370f61bdd)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:53.656903 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900e49b-1561-41e6-a779-4fdc833887ff None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:53.658720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 176/363] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 9 22:48:53 2022] GET /identity/v3/groups/d2102ce0e44a47a58cffd1e370f61bdd => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:53.659367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900e49b-1561-41e6-a779-4fdc833887ff None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:53.667456 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5fbc5-eee1-43ad-b767-09391474f472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:53.668158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5fbc5-eee1-43ad-b767-09391474f472 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:53.668387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5fbc5-eee1-43ad-b767-09391474f472 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:53.668663 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5fbc5-eee1-43ad-b767-09391474f472 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:53.751718 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5fbc5-eee1-43ad-b767-09391474f472 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:53.753187 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5fbc5-eee1-43ad-b767-09391474f472 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:53.776152 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900e49b-1561-41e6-a779-4fdc833887ff None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:53.814839 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42af3dc-0dd7-4607-9d7b-8c8eacd5d7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:53.815363 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42af3dc-0dd7-4607-9d7b-8c8eacd5d7ba None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:53.815650 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42af3dc-0dd7-4607-9d7b-8c8eacd5d7ba None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:53.815928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42af3dc-0dd7-4607-9d7b-8c8eacd5d7ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:53.816767 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42af3dc-0dd7-4607-9d7b-8c8eacd5d7ba None admin] RBAC: Adding query filter params (group.id=d2102ce0e44a47a58cffd1e370f61bdd, role.id=46ddc3cf4b124399a275bc639635e10a, scope.project.id=1a6770d749f644eea4fd1584b1503d28) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:53.817079 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42af3dc-0dd7-4607-9d7b-8c8eacd5d7ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:53.819428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42af3dc-0dd7-4607-9d7b-8c8eacd5d7ba None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:53.826189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 177/364] 10.222.0.23 () {56 vars in 1376 bytes} [Mon May 9 22:48:53 2022] GET /identity/v3/role_assignments?group.id=d2102ce0e44a47a58cffd1e370f61bdd&scope.project.id=1a6770d749f644eea4fd1584b1503d28&role.id=46ddc3cf4b124399a275bc639635e10a => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:53.827307 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42af3dc-0dd7-4607-9d7b-8c8eacd5d7ba None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:53.880472 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 188/365] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:48:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:48:53.883077 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42924869-5d68-4bb7-8a14-d010f03d9d69 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:53.883077 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42924869-5d68-4bb7-8a14-d010f03d9d69 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:53.883077 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42924869-5d68-4bb7-8a14-d010f03d9d69 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:53.885596 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 189/366] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:53.885596 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42924869-5d68-4bb7-8a14-d010f03d9d69 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:53.986341 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42af3dc-0dd7-4607-9d7b-8c8eacd5d7ba None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:54.015281 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42924869-5d68-4bb7-8a14-d010f03d9d69 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:54.030090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b0c14-893b-4e06-bac7-2a69ed6cd2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:54.030830 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b0c14-893b-4e06-bac7-2a69ed6cd2c0 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:54.031112 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b0c14-893b-4e06-bac7-2a69ed6cd2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:54.031566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b0c14-893b-4e06-bac7-2a69ed6cd2c0 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:54.032835 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b0c14-893b-4e06-bac7-2a69ed6cd2c0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:54.033178 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b0c14-893b-4e06-bac7-2a69ed6cd2c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:54.035106 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b0c14-893b-4e06-bac7-2a69ed6cd2c0 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:54.048443 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 178/367] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:54.049354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b0c14-893b-4e06-bac7-2a69ed6cd2c0 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:54.060441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469347ea-b8ff-451e-a501-80cc85d01a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:54.061116 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469347ea-b8ff-451e-a501-80cc85d01a9a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:54.061392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469347ea-b8ff-451e-a501-80cc85d01a9a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:54.061665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469347ea-b8ff-451e-a501-80cc85d01a9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:54.062556 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469347ea-b8ff-451e-a501-80cc85d01a9a None admin] RBAC: Adding query filter params (group.id=d2102ce0e44a47a58cffd1e370f61bdd, role.id=3631750e6bf34730aa3557cd90aa6108, scope.project.id=db7334eab91d41fa884c491d1364a253) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:54.062866 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469347ea-b8ff-451e-a501-80cc85d01a9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:54.064474 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469347ea-b8ff-451e-a501-80cc85d01a9a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:54.071165 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 190/368] 10.222.0.23 () {56 vars in 1376 bytes} [Mon May 9 22:48:53 2022] GET /identity/v3/role_assignments?group.id=d2102ce0e44a47a58cffd1e370f61bdd&scope.project.id=db7334eab91d41fa884c491d1364a253&role.id=3631750e6bf34730aa3557cd90aa6108 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:54.072447 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51ea8d-151f-48f4-ab02-ffd750d53267 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:54.072731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51ea8d-151f-48f4-ab02-ffd750d53267 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:54.073004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51ea8d-151f-48f4-ab02-ffd750d53267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:54.094158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-2a51ea8d-151f-48f4-ab02-ffd750d53267 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:54.094158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2a51ea8d-151f-48f4-ab02-ffd750d53267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:54.167253 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 191/369] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:54.168059 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51ea8d-151f-48f4-ab02-ffd750d53267 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:54.272705 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51ea8d-151f-48f4-ab02-ffd750d53267 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:54.286045 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b0c14-893b-4e06-bac7-2a69ed6cd2c0 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:54.312478 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ca049-27a5-4ed8-8a04-5a64772f00b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:54.313685 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ca049-27a5-4ed8-8a04-5a64772f00b6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:54.314184 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ca049-27a5-4ed8-8a04-5a64772f00b6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:54.314644 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ca049-27a5-4ed8-8a04-5a64772f00b6 None admin] PATH_INFO: `/v3/users/7a2a0b3c138448d093d12616eab013d6` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:54.319645 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ca049-27a5-4ed8-8a04-5a64772f00b6 None admin] RBAC: Authorizing `identity:get_user(user_id=7a2a0b3c138448d093d12616eab013d6)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:54.321409 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ca049-27a5-4ed8-8a04-5a64772f00b6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:54.324649 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 192/370] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:48:54 2022] GET /identity/v3/users/7a2a0b3c138448d093d12616eab013d6 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:54.325839 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ca049-27a5-4ed8-8a04-5a64772f00b6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:54.350054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644a411-ddf6-454b-a173-92c749856b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:54.351277 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644a411-ddf6-454b-a173-92c749856b9e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:54.351748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644a411-ddf6-454b-a173-92c749856b9e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:54.352205 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644a411-ddf6-454b-a173-92c749856b9e None admin] PATH_INFO: `/v3/projects/db7334eab91d41fa884c491d1364a253` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:54.358985 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a644a411-ddf6-454b-a173-92c749856b9e None admin] RBAC: Authorizing `identity:get_project(project_id=db7334eab91d41fa884c491d1364a253)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:54.360824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a644a411-ddf6-454b-a173-92c749856b9e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:54.364164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 179/371] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:54 2022] GET /identity/v3/projects/db7334eab91d41fa884c491d1364a253 => generated 302 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:54.364164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644a411-ddf6-454b-a173-92c749856b9e None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:54.514572 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644a411-ddf6-454b-a173-92c749856b9e None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:54.520254 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ca049-27a5-4ed8-8a04-5a64772f00b6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:54.557383 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aec1d4-9472-4bf5-9100-f083b07e541c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:54.558462 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aec1d4-9472-4bf5-9100-f083b07e541c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:54.558956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aec1d4-9472-4bf5-9100-f083b07e541c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:54.559436 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aec1d4-9472-4bf5-9100-f083b07e541c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:54.560461 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aec1d4-9472-4bf5-9100-f083b07e541c None admin] RBAC: Adding query filter params (role.id=949d4917fcdb41e2a938aadb277d1970, user.id=7a2a0b3c138448d093d12616eab013d6, scope.system=all) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:54.560982 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aec1d4-9472-4bf5-9100-f083b07e541c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:54.562655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aec1d4-9472-4bf5-9100-f083b07e541c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:54.576121 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 180/372] 10.222.0.23 () {56 vars in 1308 bytes} [Mon May 9 22:48:54 2022] GET /identity/v3/role_assignments?user.id=7a2a0b3c138448d093d12616eab013d6&scope.system=all&role.id=949d4917fcdb41e2a938aadb277d1970 => generated 510 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:54.577623 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aec1d4-9472-4bf5-9100-f083b07e541c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:54.606468 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a12a07-3e09-4f0d-a715-81544f245475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:54.607303 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a12a07-3e09-4f0d-a715-81544f245475 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:54.607577 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a12a07-3e09-4f0d-a715-81544f245475 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:54.607862 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a12a07-3e09-4f0d-a715-81544f245475 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:54.639141 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a12a07-3e09-4f0d-a715-81544f245475 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:54.651175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-47a12a07-3e09-4f0d-a715-81544f245475 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:48:54.651175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:54.651175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:54.651175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:54.651175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:54.651175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:54.651175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:54.651175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:54.651175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:54.651175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:54.651175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:54.651175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:54.651175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:54.651175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:54.651175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:54.651175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:54.652196 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:54.652196 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:54.652196 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:54.652196 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:54.652196 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:54.652196 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:54.652196 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:54.652196 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:54.652196 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:54.652196 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:54.652196 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:54.652196 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:54.652196 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:54.652196 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:54.652196 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:54.652196 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:54.653064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:54.653064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:54.653064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:54.653064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:54.653064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:54.653064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:54.653064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:54.653064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:54.653064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:54.653064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:54.653064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:48:54.653064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:54.653064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 193/373] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:48:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:54.658989 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a12a07-3e09-4f0d-a715-81544f245475 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:54.710620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aec1d4-9472-4bf5-9100-f083b07e541c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:54.757546 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645631a-7a03-4d4e-b054-f9ab095dbe61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:54.759038 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645631a-7a03-4d4e-b054-f9ab095dbe61 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:54.759541 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645631a-7a03-4d4e-b054-f9ab095dbe61 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:54.760197 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645631a-7a03-4d4e-b054-f9ab095dbe61 None admin] PATH_INFO: `/v3/groups/d2102ce0e44a47a58cffd1e370f61bdd` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:54.761716 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645631a-7a03-4d4e-b054-f9ab095dbe61 None admin] RBAC: Authorizing `identity:get_group(group_id=d2102ce0e44a47a58cffd1e370f61bdd)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:54.765056 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645631a-7a03-4d4e-b054-f9ab095dbe61 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:54.767336 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 181/374] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 9 22:48:54 2022] GET /identity/v3/groups/d2102ce0e44a47a58cffd1e370f61bdd => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:54.773496 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645631a-7a03-4d4e-b054-f9ab095dbe61 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:54.869805 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a12a07-3e09-4f0d-a715-81544f245475 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:54.940574 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645631a-7a03-4d4e-b054-f9ab095dbe61 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:54.943705 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263900f-f6da-4968-9441-02678a62426e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:54.944380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263900f-f6da-4968-9441-02678a62426e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:54.944655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263900f-f6da-4968-9441-02678a62426e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:54.944922 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263900f-f6da-4968-9441-02678a62426e None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:54.945670 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263900f-f6da-4968-9441-02678a62426e None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:54.945967 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263900f-f6da-4968-9441-02678a62426e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:54.947368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263900f-f6da-4968-9441-02678a62426e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:54.966011 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 194/375] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:54.972551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263900f-f6da-4968-9441-02678a62426e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:54.993754 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b14a877-58a8-4196-96ba-c2e7cf90c8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:54.994488 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14a877-58a8-4196-96ba-c2e7cf90c8e7 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:54.994774 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14a877-58a8-4196-96ba-c2e7cf90c8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:54.995049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14a877-58a8-4196-96ba-c2e7cf90c8e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:54.995888 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b14a877-58a8-4196-96ba-c2e7cf90c8e7 None admin] RBAC: Adding query filter params (group.id=d2102ce0e44a47a58cffd1e370f61bdd, role.id=46ddc3cf4b124399a275bc639635e10a, scope.project.id=db7334eab91d41fa884c491d1364a253) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:54.996196 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b14a877-58a8-4196-96ba-c2e7cf90c8e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:54.997645 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b14a877-58a8-4196-96ba-c2e7cf90c8e7 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:55.006905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 182/376] 10.222.0.23 () {56 vars in 1376 bytes} [Mon May 9 22:48:54 2022] GET /identity/v3/role_assignments?group.id=d2102ce0e44a47a58cffd1e370f61bdd&scope.project.id=db7334eab91d41fa884c491d1364a253&role.id=46ddc3cf4b124399a275bc639635e10a => generated 260 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:55.103435 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263900f-f6da-4968-9441-02678a62426e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:55.141518 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7317018a-6c90-4c79-bf21-6bdac8003c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:55.142192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317018a-6c90-4c79-bf21-6bdac8003c4d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:55.142468 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317018a-6c90-4c79-bf21-6bdac8003c4d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:55.142737 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317018a-6c90-4c79-bf21-6bdac8003c4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:55.218710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7317018a-6c90-4c79-bf21-6bdac8003c4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:55.220194 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7317018a-6c90-4c79-bf21-6bdac8003c4d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:55.326311 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 195/377] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:48:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 22:48:55.331704 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b14a877-58a8-4196-96ba-c2e7cf90c8e7 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:55.448480 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b14a877-58a8-4196-96ba-c2e7cf90c8e7 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:55.491454 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2f83e-878a-4ff8-bbe2-d12d315e41e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:55.492317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2f83e-878a-4ff8-bbe2-d12d315e41e5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:55.492594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2f83e-878a-4ff8-bbe2-d12d315e41e5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:55.492863 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2f83e-878a-4ff8-bbe2-d12d315e41e5 None admin] PATH_INFO: `/v3/projects/4202d89a17374bc0a3d992386d254d4c` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:55.494054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2f83e-878a-4ff8-bbe2-d12d315e41e5 None admin] RBAC: Authorizing `identity:get_project(project_id=4202d89a17374bc0a3d992386d254d4c)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:55.498158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2f83e-878a-4ff8-bbe2-d12d315e41e5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:55.498314 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b98af-a966-4b61-adc9-260e0814b0d6 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:55.498314 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b98af-a966-4b61-adc9-260e0814b0d6 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:55.498469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 183/378] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:55 2022] GET /identity/v3/projects/4202d89a17374bc0a3d992386d254d4c => generated 344 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:55.498539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b98af-a966-4b61-adc9-260e0814b0d6 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:55.499573 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 196/379] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:55.502600 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2f83e-878a-4ff8-bbe2-d12d315e41e5 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:55.505924 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d4df9-9d72-4e90-b992-2232055ea223 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:55.506619 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d4df9-9d72-4e90-b992-2232055ea223 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:55.507096 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d4df9-9d72-4e90-b992-2232055ea223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:55.530454 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-128d4df9-9d72-4e90-b992-2232055ea223 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:55.531983 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-128d4df9-9d72-4e90-b992-2232055ea223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:55.641624 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 197/380] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:55.669572 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2f83e-878a-4ff8-bbe2-d12d315e41e5 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:55.679751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d4df9-9d72-4e90-b992-2232055ea223 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:55.703906 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68288325-343d-40fc-a39f-00b5d7f0764b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:55.704689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68288325-343d-40fc-a39f-00b5d7f0764b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:55.704971 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68288325-343d-40fc-a39f-00b5d7f0764b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:55.705249 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68288325-343d-40fc-a39f-00b5d7f0764b None admin] PATH_INFO: `/v3/groups/57bf5890247b4a7995c02be731a983ce` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:55.712119 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68288325-343d-40fc-a39f-00b5d7f0764b None admin] RBAC: Authorizing `identity:get_group(group_id=57bf5890247b4a7995c02be731a983ce)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:55.713575 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68288325-343d-40fc-a39f-00b5d7f0764b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:55.717451 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 184/381] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 9 22:48:55 2022] GET /identity/v3/groups/57bf5890247b4a7995c02be731a983ce => generated 229 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:55.721805 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68288325-343d-40fc-a39f-00b5d7f0764b None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:55.817312 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d4df9-9d72-4e90-b992-2232055ea223 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:55.871605 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dbd533-4d06-476f-905b-9278c92b3c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:55.872454 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbd533-4d06-476f-905b-9278c92b3c9b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:55.872745 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbd533-4d06-476f-905b-9278c92b3c9b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:55.873020 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbd533-4d06-476f-905b-9278c92b3c9b None admin] PATH_INFO: `/v3/roles/member` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:55.894060 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dbd533-4d06-476f-905b-9278c92b3c9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:55.909041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-c5dbd533-4d06-476f-905b-9278c92b3c9b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:55.909041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:55.909041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:55.909041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:55.909041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:55.909041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:55.909041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:55.909041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:55.909041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:55.909041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:55.909041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:55.909041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:55.909041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:55.909041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:55.909041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:55.909041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:55.909990 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:55.909990 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:55.909990 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:55.909990 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:55.909990 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:55.909990 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:55.909990 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:55.909990 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:55.909990 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:55.909990 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:55.909990 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:55.909990 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:55.909990 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:55.909990 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:55.909990 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:55.909990 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:55.910812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:55.910812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:55.910812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:55.910812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:55.910812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:55.910812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:55.910812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:55.910812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:55.910812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:55.910812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:55.910812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:55.910812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:55.910812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 198/382] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:55.914217 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68288325-343d-40fc-a39f-00b5d7f0764b None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:55.916637 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dbd533-4d06-476f-905b-9278c92b3c9b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:55.977588 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528ed72-6210-4aff-af09-98f58353085d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:55.978189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528ed72-6210-4aff-af09-98f58353085d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:55.978471 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528ed72-6210-4aff-af09-98f58353085d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:55.978741 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528ed72-6210-4aff-af09-98f58353085d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:55.979563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6528ed72-6210-4aff-af09-98f58353085d None admin] RBAC: Adding query filter params (group.id=57bf5890247b4a7995c02be731a983ce, role.id=bda550492f4f4c5c83f52fa89fc18a7d, scope.project.id=4202d89a17374bc0a3d992386d254d4c) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:55.979869 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6528ed72-6210-4aff-af09-98f58353085d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:55.986166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6528ed72-6210-4aff-af09-98f58353085d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:55.996078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 185/383] 10.222.0.23 () {56 vars in 1376 bytes} [Mon May 9 22:48:55 2022] GET /identity/v3/role_assignments?group.id=57bf5890247b4a7995c02be731a983ce&scope.project.id=4202d89a17374bc0a3d992386d254d4c&role.id=bda550492f4f4c5c83f52fa89fc18a7d => generated 260 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:56.055555 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dbd533-4d06-476f-905b-9278c92b3c9b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:56.093544 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48351b5-ac69-4069-b7c2-abb1c0d7ae50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:56.094251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48351b5-ac69-4069-b7c2-abb1c0d7ae50 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:56.094532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48351b5-ac69-4069-b7c2-abb1c0d7ae50 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:56.094800 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48351b5-ac69-4069-b7c2-abb1c0d7ae50 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:56.095555 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48351b5-ac69-4069-b7c2-abb1c0d7ae50 None admin] RBAC: Adding query filter params (name=member) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:56.095854 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48351b5-ac69-4069-b7c2-abb1c0d7ae50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:56.097249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48351b5-ac69-4069-b7c2-abb1c0d7ae50 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:56.106651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 199/384] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:56.112913 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528ed72-6210-4aff-af09-98f58353085d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:56.237116 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528ed72-6210-4aff-af09-98f58353085d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:56.275090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b5c85-7724-4074-a3ff-52f52219a27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:56.275999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b5c85-7724-4074-a3ff-52f52219a27d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:56.276272 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b5c85-7724-4074-a3ff-52f52219a27d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:56.276544 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b5c85-7724-4074-a3ff-52f52219a27d None admin] PATH_INFO: `/v3/groups/d2102ce0e44a47a58cffd1e370f61bdd` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:56.282802 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b5c85-7724-4074-a3ff-52f52219a27d None admin] RBAC: Authorizing `identity:get_group(group_id=d2102ce0e44a47a58cffd1e370f61bdd)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:56.284251 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b5c85-7724-4074-a3ff-52f52219a27d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:56.286209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 186/385] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 9 22:48:56 2022] GET /identity/v3/groups/d2102ce0e44a47a58cffd1e370f61bdd => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:56.293439 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48351b5-ac69-4069-b7c2-abb1c0d7ae50 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:56.418758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48351b5-ac69-4069-b7c2-abb1c0d7ae50 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:56.468720 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca6e2e-44df-4aa3-9507-8e2d8324957b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:56.469966 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca6e2e-44df-4aa3-9507-8e2d8324957b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:56.470464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca6e2e-44df-4aa3-9507-8e2d8324957b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:56.470947 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca6e2e-44df-4aa3-9507-8e2d8324957b None admin] PATH_INFO: `/v3/projects/1a6770d749f644eea4fd1584b1503d28` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:56.472507 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca6e2e-44df-4aa3-9507-8e2d8324957b None admin] RBAC: Authorizing `identity:get_project(project_id=1a6770d749f644eea4fd1584b1503d28)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:56.474312 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca6e2e-44df-4aa3-9507-8e2d8324957b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:56.476534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 200/386] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:56 2022] GET /identity/v3/projects/1a6770d749f644eea4fd1584b1503d28 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:56.481886 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b5c85-7724-4074-a3ff-52f52219a27d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:56.617968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b5c85-7724-4074-a3ff-52f52219a27d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:56.659238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f488ef0-6b44-4fd3-b222-bbcd560c8b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:56.660134 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f488ef0-6b44-4fd3-b222-bbcd560c8b93 None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:56.660407 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f488ef0-6b44-4fd3-b222-bbcd560c8b93 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:56.660673 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f488ef0-6b44-4fd3-b222-bbcd560c8b93 None admin] PATH_INFO: `/v3/projects/1a6770d749f644eea4fd1584b1503d28/groups/d2102ce0e44a47a58cffd1e370f61bdd/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:56.667847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f488ef0-6b44-4fd3-b222-bbcd560c8b93 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a6770d749f644eea4fd1584b1503d28, group_id=d2102ce0e44a47a58cffd1e370f61bdd, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:56.669333 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f488ef0-6b44-4fd3-b222-bbcd560c8b93 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:56.702370 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 187/387] 10.222.0.23 () {58 vars in 1442 bytes} [Mon May 9 22:48:56 2022] PUT /identity/v3/projects/1a6770d749f644eea4fd1584b1503d28/groups/d2102ce0e44a47a58cffd1e370f61bdd/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:48:56.887204 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8d680-f61e-4ae2-9236-8dc7ef7fda37 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:56.887549 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8d680-f61e-4ae2-9236-8dc7ef7fda37 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:56.887667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8d680-f61e-4ae2-9236-8dc7ef7fda37 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:56.889270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 201/388] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:56.894784 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1581a-88d4-46e3-8504-d679b175c53f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:56.895071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1581a-88d4-46e3-8504-d679b175c53f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:56.895339 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1581a-88d4-46e3-8504-d679b175c53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:56.918587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-44d1581a-88d4-46e3-8504-d679b175c53f None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:56.919769 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-44d1581a-88d4-46e3-8504-d679b175c53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:57.005256 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 188/389] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:57.041309 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8d680-f61e-4ae2-9236-8dc7ef7fda37 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:57.163298 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8d680-f61e-4ae2-9236-8dc7ef7fda37 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:57.202751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c941c5-8f98-4e57-80ac-fe5431a02e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:57.203582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c941c5-8f98-4e57-80ac-fe5431a02e0e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:57.203863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c941c5-8f98-4e57-80ac-fe5431a02e0e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:57.204138 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c941c5-8f98-4e57-80ac-fe5431a02e0e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:57.219015 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c941c5-8f98-4e57-80ac-fe5431a02e0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:57.227923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-a8c941c5-8f98-4e57-80ac-fe5431a02e0e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:48:57.227923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:57.227923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:57.227923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:57.227923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:57.227923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:57.227923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:57.227923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:57.227923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:57.227923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:57.227923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:57.227923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:57.227923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:57.227923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:57.227923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:57.227923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:57.228821 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:57.228821 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:57.228821 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:57.228821 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:57.228821 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:57.228821 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:57.228821 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:57.228821 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:57.228821 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:57.228821 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:57.228821 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:57.228821 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:57.228821 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:57.228821 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:57.228821 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:57.228821 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:57.230297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:57.230297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:57.230297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:57.230297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:57.230297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:57.230297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:57.230297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:57.230297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:57.230297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:57.230297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:57.230297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:48:57.230297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:57.230297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 202/390] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 9 22:48:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:57.235184 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1581a-88d4-46e3-8504-d679b175c53f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:57.390274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951223f3-b9e2-4f43-b766-b50b146d7ee5 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:57.390584 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951223f3-b9e2-4f43-b766-b50b146d7ee5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:57.390845 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951223f3-b9e2-4f43-b766-b50b146d7ee5 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:57.392445 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 203/391] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:57.397623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e560b0-255c-4874-a3fd-4274bd4df9ad None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:57.397913 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e560b0-255c-4874-a3fd-4274bd4df9ad None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:57.398211 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e560b0-255c-4874-a3fd-4274bd4df9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:57.398428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1581a-88d4-46e3-8504-d679b175c53f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:57.419669 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d5e560b0-255c-4874-a3fd-4274bd4df9ad None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:57.420775 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d5e560b0-255c-4874-a3fd-4274bd4df9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:57.451367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc1c7c-4d54-4ae9-8211-eb06e0c0d12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:57.452368 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc1c7c-4d54-4ae9-8211-eb06e0c0d12f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:57.452653 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc1c7c-4d54-4ae9-8211-eb06e0c0d12f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:57.453186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc1c7c-4d54-4ae9-8211-eb06e0c0d12f None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:57.453934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc1c7c-4d54-4ae9-8211-eb06e0c0d12f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:57.454413 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc1c7c-4d54-4ae9-8211-eb06e0c0d12f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:57.456404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc1c7c-4d54-4ae9-8211-eb06e0c0d12f None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:57.470612 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 189/392] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:48:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:57.478896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc1c7c-4d54-4ae9-8211-eb06e0c0d12f None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:57.509155 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 204/393] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:48:57.559371 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e560b0-255c-4874-a3fd-4274bd4df9ad None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:57.610103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc1c7c-4d54-4ae9-8211-eb06e0c0d12f None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:57.654810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a6601-c70d-4453-8962-a976ffae4656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:57.655680 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a6601-c70d-4453-8962-a976ffae4656 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:57.656010 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a6601-c70d-4453-8962-a976ffae4656 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:57.656286 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a6601-c70d-4453-8962-a976ffae4656 None admin] PATH_INFO: `/v3/groups/d2102ce0e44a47a58cffd1e370f61bdd` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:57.657454 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a6601-c70d-4453-8962-a976ffae4656 None admin] RBAC: Authorizing `identity:get_group(group_id=d2102ce0e44a47a58cffd1e370f61bdd)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:57.658918 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a6601-c70d-4453-8962-a976ffae4656 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:57.660821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 190/394] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 9 22:48:57 2022] GET /identity/v3/groups/d2102ce0e44a47a58cffd1e370f61bdd => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:57.666723 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a6601-c70d-4453-8962-a976ffae4656 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:57.694212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e560b0-255c-4874-a3fd-4274bd4df9ad None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:57.729151 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d231408-43b5-47d1-89a9-8397f6cb855e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:57.729937 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d231408-43b5-47d1-89a9-8397f6cb855e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:57.730235 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d231408-43b5-47d1-89a9-8397f6cb855e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:57.730504 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d231408-43b5-47d1-89a9-8397f6cb855e None admin] PATH_INFO: `/v3/roles/member` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:57.742001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d231408-43b5-47d1-89a9-8397f6cb855e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:57.749689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-8d231408-43b5-47d1-89a9-8397f6cb855e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:57.749689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:57.749689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:57.749689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:57.749689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:57.749689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:57.749689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:57.749689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:57.749689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:57.749689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:57.749689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:57.749689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:57.749689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:57.749689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:57.749689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:57.749689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:57.750602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:57.750602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:57.750602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:57.750602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:57.750602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:57.750602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:57.750602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:57.750602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:57.750602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:57.750602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:57.750602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:57.750602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:57.750602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:57.750602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:57.750602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:57.750602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:57.751880 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:57.751880 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:57.751880 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:57.751880 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:57.751880 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:57.751880 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:57.751880 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:57.751880 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:57.751880 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:57.751880 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:57.751880 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 22:48:57.751880 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:57.751880 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 205/395] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:48:57.771142 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a6601-c70d-4453-8962-a976ffae4656 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:57.775999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d231408-43b5-47d1-89a9-8397f6cb855e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:57.807655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda8395-9e9c-4652-8cb6-eada5e5bfeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:57.808575 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda8395-9e9c-4652-8cb6-eada5e5bfeb8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:57.808852 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda8395-9e9c-4652-8cb6-eada5e5bfeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:57.809178 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda8395-9e9c-4652-8cb6-eada5e5bfeb8 None admin] PATH_INFO: `/v3/projects/1a6770d749f644eea4fd1584b1503d28` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:57.810420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda8395-9e9c-4652-8cb6-eada5e5bfeb8 None admin] RBAC: Authorizing `identity:get_project(project_id=1a6770d749f644eea4fd1584b1503d28)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:57.811945 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda8395-9e9c-4652-8cb6-eada5e5bfeb8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:57.813979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 191/396] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:57 2022] GET /identity/v3/projects/1a6770d749f644eea4fd1584b1503d28 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:57.820344 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda8395-9e9c-4652-8cb6-eada5e5bfeb8 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:57.930649 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda8395-9e9c-4652-8cb6-eada5e5bfeb8 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:57.968464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceebaa7-b728-435d-b578-d89d44963549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:57.969761 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceebaa7-b728-435d-b578-d89d44963549 None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:57.970080 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceebaa7-b728-435d-b578-d89d44963549 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:57.970381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceebaa7-b728-435d-b578-d89d44963549 None admin] PATH_INFO: `/v3/projects/1a6770d749f644eea4fd1584b1503d28/groups/d2102ce0e44a47a58cffd1e370f61bdd/roles/46ddc3cf4b124399a275bc639635e10a` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:57.972812 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceebaa7-b728-435d-b578-d89d44963549 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a6770d749f644eea4fd1584b1503d28, group_id=d2102ce0e44a47a58cffd1e370f61bdd, role_id=46ddc3cf4b124399a275bc639635e10a)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:57.974294 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceebaa7-b728-435d-b578-d89d44963549 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:57.990181 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d231408-43b5-47d1-89a9-8397f6cb855e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:58.000144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 192/397] 10.222.0.23 () {58 vars in 1442 bytes} [Mon May 9 22:48:57 2022] PUT /identity/v3/projects/1a6770d749f644eea4fd1584b1503d28/groups/d2102ce0e44a47a58cffd1e370f61bdd/roles/46ddc3cf4b124399a275bc639635e10a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 22:48:58.044846 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcfe8df-2a68-4709-b4a1-22c0bd9118d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:58.045537 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcfe8df-2a68-4709-b4a1-22c0bd9118d6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:58.045818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcfe8df-2a68-4709-b4a1-22c0bd9118d6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:58.046092 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcfe8df-2a68-4709-b4a1-22c0bd9118d6 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:58.046982 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcfe8df-2a68-4709-b4a1-22c0bd9118d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:58.047293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcfe8df-2a68-4709-b4a1-22c0bd9118d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:58.048712 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcfe8df-2a68-4709-b4a1-22c0bd9118d6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:58.061094 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 206/398] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:48:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:58.067271 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceebaa7-b728-435d-b578-d89d44963549 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:58.186578 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceebaa7-b728-435d-b578-d89d44963549 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:58.227714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e34159-7a0c-42f4-a897-c18677c55c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:58.228492 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e34159-7a0c-42f4-a897-c18677c55c9a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:58.228769 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e34159-7a0c-42f4-a897-c18677c55c9a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:58.229042 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e34159-7a0c-42f4-a897-c18677c55c9a None admin] PATH_INFO: `/v3/groups/d2102ce0e44a47a58cffd1e370f61bdd` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:58.230410 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e34159-7a0c-42f4-a897-c18677c55c9a None admin] RBAC: Authorizing `identity:get_group(group_id=d2102ce0e44a47a58cffd1e370f61bdd)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:58.231865 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e34159-7a0c-42f4-a897-c18677c55c9a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:58.233793 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 193/399] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 9 22:48:58 2022] GET /identity/v3/groups/d2102ce0e44a47a58cffd1e370f61bdd => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:58.239553 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcfe8df-2a68-4709-b4a1-22c0bd9118d6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:58.349399 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcfe8df-2a68-4709-b4a1-22c0bd9118d6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:58.385128 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c6f17-daff-4734-a456-b0dfe862ec7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:58.386041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c6f17-daff-4734-a456-b0dfe862ec7a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:58.386352 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c6f17-daff-4734-a456-b0dfe862ec7a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:58.386623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c6f17-daff-4734-a456-b0dfe862ec7a None admin] PATH_INFO: `/v3/projects/db7334eab91d41fa884c491d1364a253` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:58.387885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c6f17-daff-4734-a456-b0dfe862ec7a None admin] RBAC: Authorizing `identity:get_project(project_id=db7334eab91d41fa884c491d1364a253)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:58.389342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c6f17-daff-4734-a456-b0dfe862ec7a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:58.391466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 207/400] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:58 2022] GET /identity/v3/projects/db7334eab91d41fa884c491d1364a253 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:58.397919 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e34159-7a0c-42f4-a897-c18677c55c9a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:58.428328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c862df7-9d0a-4e68-8fbd-a368207b21f5 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:58.428328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c862df7-9d0a-4e68-8fbd-a368207b21f5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:58.428328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c862df7-9d0a-4e68-8fbd-a368207b21f5 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:58.428328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 208/401] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:48:58.437106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d9929-78c8-438c-8939-2cf714c8ed6f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:58.437106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d9929-78c8-438c-8939-2cf714c8ed6f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:58.437106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d9929-78c8-438c-8939-2cf714c8ed6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:58.463045 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-908d9929-78c8-438c-8939-2cf714c8ed6f None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:58.464624 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-908d9929-78c8-438c-8939-2cf714c8ed6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:58.531198 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e34159-7a0c-42f4-a897-c18677c55c9a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:58.558630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 209/402] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:48:58.571367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837fdf3-45e8-4a9b-b7d1-f12db31db9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:58.572300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837fdf3-45e8-4a9b-b7d1-f12db31db9e8 None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:58.572579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837fdf3-45e8-4a9b-b7d1-f12db31db9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:58.572851 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837fdf3-45e8-4a9b-b7d1-f12db31db9e8 None admin] PATH_INFO: `/v3/projects/db7334eab91d41fa884c491d1364a253/groups/d2102ce0e44a47a58cffd1e370f61bdd/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:58.574823 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837fdf3-45e8-4a9b-b7d1-f12db31db9e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=db7334eab91d41fa884c491d1364a253, group_id=d2102ce0e44a47a58cffd1e370f61bdd, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:58.576355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837fdf3-45e8-4a9b-b7d1-f12db31db9e8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:58.591673 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 194/403] 10.222.0.23 () {58 vars in 1442 bytes} [Mon May 9 22:48:58 2022] PUT /identity/v3/projects/db7334eab91d41fa884c491d1364a253/groups/d2102ce0e44a47a58cffd1e370f61bdd/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:48:58.597307 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d9929-78c8-438c-8939-2cf714c8ed6f None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:58.660615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b0cbc-475c-4470-b182-b7d9a7686fbb None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:58.661515 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b0cbc-475c-4470-b182-b7d9a7686fbb None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:58.661799 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b0cbc-475c-4470-b182-b7d9a7686fbb None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:58.663941 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 195/404] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:48:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:48:58.669424 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67850caa-fbfc-4714-abbc-beba789a2f41 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:58.669713 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67850caa-fbfc-4714-abbc-beba789a2f41 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:58.669989 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67850caa-fbfc-4714-abbc-beba789a2f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:58.703479 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-67850caa-fbfc-4714-abbc-beba789a2f41 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:48:58.706970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-67850caa-fbfc-4714-abbc-beba789a2f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:48:58.798094 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d9929-78c8-438c-8939-2cf714c8ed6f None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:58.810743 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 196/405] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:48:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:48:58.837163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac35b95-d0d0-427d-b630-66c9fa646533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:58.837976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac35b95-d0d0-427d-b630-66c9fa646533 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:58.838312 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac35b95-d0d0-427d-b630-66c9fa646533 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:58.838583 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac35b95-d0d0-427d-b630-66c9fa646533 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:58.842455 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67850caa-fbfc-4714-abbc-beba789a2f41 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:58.866887 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac35b95-d0d0-427d-b630-66c9fa646533 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:58.876327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-8ac35b95-d0d0-427d-b630-66c9fa646533 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:48:58.876327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:58.876327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:58.876327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:58.876327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:58.876327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:58.876327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:58.876327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:58.876327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:58.876327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:58.876327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:58.876327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:58.876327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:58.876327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:58.876327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:58.876327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:58.878300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:58.878300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:58.878300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:58.878300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:58.878300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:58.878300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:58.878300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:58.878300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:58.878300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:58.878300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:58.878300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:58.878300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:48:58.878300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:58.878300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:58.878300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:58.878300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:58.879279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:58.879279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:58.879279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:58.879279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:58.879279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:58.879279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:58.879279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:58.879279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:58.879279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:58.879279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:58.879279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:48:58.879279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:48:58.879279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 210/406] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 9 22:48:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:48:58.887394 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac35b95-d0d0-427d-b630-66c9fa646533 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:58.989066 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67850caa-fbfc-4714-abbc-beba789a2f41 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:59.045214 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb054b04-823f-4519-8301-4f2e8b59dd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:59.046014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb054b04-823f-4519-8301-4f2e8b59dd60 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:59.046327 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb054b04-823f-4519-8301-4f2e8b59dd60 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:59.046775 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb054b04-823f-4519-8301-4f2e8b59dd60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:59.060341 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb054b04-823f-4519-8301-4f2e8b59dd60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:48:59.071559 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac35b95-d0d0-427d-b630-66c9fa646533 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:59.071810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-cb054b04-823f-4519-8301-4f2e8b59dd60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:48:59.071810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:48:59.071810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:48:59.071810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:48:59.071810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:48:59.071810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:48:59.071810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:48:59.071810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:48:59.071810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:48:59.071810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:48:59.071810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:48:59.071810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:48:59.071810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:48:59.071810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:48:59.071810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:48:59.071810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:48:59.073811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:48:59.073811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:48:59.073811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:48:59.073811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:48:59.073811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:48:59.073811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:48:59.073811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:48:59.073811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:48:59.073811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:48:59.073811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:48:59.073811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:48:59.073811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:48:59.073811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:48:59.073811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:48:59.073811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:48:59.073811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:48:59.079282 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:48:59.079282 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:48:59.079282 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:48:59.079282 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:48:59.079282 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:48:59.079282 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:48:59.079282 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:48:59.079282 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:48:59.079282 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:48:59.079282 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:48:59.079282 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:48:59.079282 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:48:59.079282 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 197/407] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:48:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:48:59.080968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb054b04-823f-4519-8301-4f2e8b59dd60 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:59.135847 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe419cc-3c72-4b82-9a44-a2c5c12df5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:59.136567 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe419cc-3c72-4b82-9a44-a2c5c12df5f0 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:59.136840 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe419cc-3c72-4b82-9a44-a2c5c12df5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:59.137106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe419cc-3c72-4b82-9a44-a2c5c12df5f0 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:59.137947 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe419cc-3c72-4b82-9a44-a2c5c12df5f0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:59.138279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe419cc-3c72-4b82-9a44-a2c5c12df5f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:59.139639 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe419cc-3c72-4b82-9a44-a2c5c12df5f0 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:59.150982 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 211/408] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:48:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:59.156659 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe419cc-3c72-4b82-9a44-a2c5c12df5f0 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:59.268034 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb054b04-823f-4519-8301-4f2e8b59dd60 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:59.276839 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe419cc-3c72-4b82-9a44-a2c5c12df5f0 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:59.317193 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bd106-701a-4398-a1a3-c52f143844c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:59.317975 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bd106-701a-4398-a1a3-c52f143844c9 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:59.318297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bd106-701a-4398-a1a3-c52f143844c9 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:59.318646 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bd106-701a-4398-a1a3-c52f143844c9 None admin] PATH_INFO: `/v3/groups/d2102ce0e44a47a58cffd1e370f61bdd` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:59.321430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653bd106-701a-4398-a1a3-c52f143844c9 None admin] RBAC: Authorizing `identity:get_group(group_id=d2102ce0e44a47a58cffd1e370f61bdd)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:59.323150 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653bd106-701a-4398-a1a3-c52f143844c9 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:59.324871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 212/409] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 9 22:48:59 2022] GET /identity/v3/groups/d2102ce0e44a47a58cffd1e370f61bdd => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:59.330419 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bd106-701a-4398-a1a3-c52f143844c9 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:59.338901 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e507f34-8736-4958-8758-9cda8b3fe7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:59.339790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e507f34-8736-4958-8758-9cda8b3fe7e7 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:59.340245 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e507f34-8736-4958-8758-9cda8b3fe7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:59.340666 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e507f34-8736-4958-8758-9cda8b3fe7e7 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:59.341630 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e507f34-8736-4958-8758-9cda8b3fe7e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:48:59.342096 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e507f34-8736-4958-8758-9cda8b3fe7e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:59.343690 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e507f34-8736-4958-8758-9cda8b3fe7e7 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:59.366062 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 198/410] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:48:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:59.371408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e507f34-8736-4958-8758-9cda8b3fe7e7 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:59.540515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bd106-701a-4398-a1a3-c52f143844c9 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:59.576018 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e507f34-8736-4958-8758-9cda8b3fe7e7 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:59.597411 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fdbbc-1b3d-4d9e-97e8-418349769b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:59.598574 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fdbbc-1b3d-4d9e-97e8-418349769b0b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:59.599297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fdbbc-1b3d-4d9e-97e8-418349769b0b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:59.600265 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fdbbc-1b3d-4d9e-97e8-418349769b0b None admin] PATH_INFO: `/v3/projects/db7334eab91d41fa884c491d1364a253` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:59.601799 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0fdbbc-1b3d-4d9e-97e8-418349769b0b None admin] RBAC: Authorizing `identity:get_project(project_id=db7334eab91d41fa884c491d1364a253)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:59.603415 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0fdbbc-1b3d-4d9e-97e8-418349769b0b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:59.605565 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 213/411] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:59 2022] GET /identity/v3/projects/db7334eab91d41fa884c491d1364a253 => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:59.619054 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fdbbc-1b3d-4d9e-97e8-418349769b0b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:59.647427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93571bd8-9975-45d6-8d7b-4f973ec7d7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:59.648289 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93571bd8-9975-45d6-8d7b-4f973ec7d7ea None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:59.648568 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93571bd8-9975-45d6-8d7b-4f973ec7d7ea None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:59.648842 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93571bd8-9975-45d6-8d7b-4f973ec7d7ea None admin] PATH_INFO: `/v3/groups/57bf5890247b4a7995c02be731a983ce` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:59.654831 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93571bd8-9975-45d6-8d7b-4f973ec7d7ea None admin] RBAC: Authorizing `identity:get_group(group_id=57bf5890247b4a7995c02be731a983ce)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:59.656709 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93571bd8-9975-45d6-8d7b-4f973ec7d7ea None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:59.658007 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 199/412] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 9 22:48:59 2022] GET /identity/v3/groups/57bf5890247b4a7995c02be731a983ce => generated 229 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:48:59.663815 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93571bd8-9975-45d6-8d7b-4f973ec7d7ea None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:48:59.726614 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fdbbc-1b3d-4d9e-97e8-418349769b0b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:59.762976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9cd7d-e9fe-4227-b296-2ccc85150924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:59.764263 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9cd7d-e9fe-4227-b296-2ccc85150924 None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:59.764755 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9cd7d-e9fe-4227-b296-2ccc85150924 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:59.765220 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9cd7d-e9fe-4227-b296-2ccc85150924 None admin] PATH_INFO: `/v3/projects/db7334eab91d41fa884c491d1364a253/groups/d2102ce0e44a47a58cffd1e370f61bdd/roles/46ddc3cf4b124399a275bc639635e10a` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:59.767845 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9cd7d-e9fe-4227-b296-2ccc85150924 None admin] RBAC: Authorizing `identity:create_grant(project_id=db7334eab91d41fa884c491d1364a253, group_id=d2102ce0e44a47a58cffd1e370f61bdd, role_id=46ddc3cf4b124399a275bc639635e10a)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:59.769604 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9cd7d-e9fe-4227-b296-2ccc85150924 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:59.790274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 214/413] 10.222.0.23 () {58 vars in 1442 bytes} [Mon May 9 22:48:59 2022] PUT /identity/v3/projects/db7334eab91d41fa884c491d1364a253/groups/d2102ce0e44a47a58cffd1e370f61bdd/roles/46ddc3cf4b124399a275bc639635e10a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:48:59.835973 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93571bd8-9975-45d6-8d7b-4f973ec7d7ea None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:48:59.877105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795cf71-e690-4946-9136-6745eeb53076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:48:59.878294 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795cf71-e690-4946-9136-6745eeb53076 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:48:59.878786 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795cf71-e690-4946-9136-6745eeb53076 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:48:59.879250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795cf71-e690-4946-9136-6745eeb53076 None admin] PATH_INFO: `/v3/projects/4202d89a17374bc0a3d992386d254d4c` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:48:59.880612 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795cf71-e690-4946-9136-6745eeb53076 None admin] RBAC: Authorizing `identity:get_project(project_id=4202d89a17374bc0a3d992386d254d4c)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:48:59.882349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795cf71-e690-4946-9136-6745eeb53076 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:48:59.884328 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 200/414] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:48:59 2022] GET /identity/v3/projects/4202d89a17374bc0a3d992386d254d4c => generated 344 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:48:59.890182 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9cd7d-e9fe-4227-b296-2ccc85150924 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:00.000051 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9cd7d-e9fe-4227-b296-2ccc85150924 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:00.008029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3667851-d404-41c3-99ed-1a831e0c40a4 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:00.008611 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3667851-d404-41c3-99ed-1a831e0c40a4 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:00.009098 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3667851-d404-41c3-99ed-1a831e0c40a4 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:00.011461 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 201/415] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:49:00.023738 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8948990-60c4-4e8c-9232-f0f471f1baf3 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:00.024919 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8948990-60c4-4e8c-9232-f0f471f1baf3 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:00.025413 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8948990-60c4-4e8c-9232-f0f471f1baf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:00.057126 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e8948990-60c4-4e8c-9232-f0f471f1baf3 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:00.059041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352874de-3853-46f5-b02f-c0bb6c4b2a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:00.059187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e8948990-60c4-4e8c-9232-f0f471f1baf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:00.060452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352874de-3853-46f5-b02f-c0bb6c4b2a2e None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:00.060995 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352874de-3853-46f5-b02f-c0bb6c4b2a2e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:00.061456 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352874de-3853-46f5-b02f-c0bb6c4b2a2e None admin] PATH_INFO: `/v3/projects/4202d89a17374bc0a3d992386d254d4c/groups/57bf5890247b4a7995c02be731a983ce/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:00.063841 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352874de-3853-46f5-b02f-c0bb6c4b2a2e None admin] RBAC: Authorizing `identity:create_grant(project_id=4202d89a17374bc0a3d992386d254d4c, group_id=57bf5890247b4a7995c02be731a983ce, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:00.065564 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352874de-3853-46f5-b02f-c0bb6c4b2a2e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:00.095390 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 215/416] 10.222.0.23 () {58 vars in 1442 bytes} [Mon May 9 22:48:59 2022] PUT /identity/v3/projects/4202d89a17374bc0a3d992386d254d4c/groups/57bf5890247b4a7995c02be731a983ce/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:49:00.158788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 202/417] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:49:00.182502 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352874de-3853-46f5-b02f-c0bb6c4b2a2e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:00.303383 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352874de-3853-46f5-b02f-c0bb6c4b2a2e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:00.346982 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2572ed-4f49-4bd9-aa8d-ad07ae9073a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:00.348720 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2572ed-4f49-4bd9-aa8d-ad07ae9073a6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:00.349422 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2572ed-4f49-4bd9-aa8d-ad07ae9073a6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:00.350216 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2572ed-4f49-4bd9-aa8d-ad07ae9073a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:00.367979 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2572ed-4f49-4bd9-aa8d-ad07ae9073a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:00.376236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-ce2572ed-4f49-4bd9-aa8d-ad07ae9073a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 22:49:00.376236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:00.376236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:00.376236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:00.376236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:00.376236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:00.376236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:00.376236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:00.376236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:00.376236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:00.376236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:00.376236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:00.376236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:00.376236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:00.376236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:00.376236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:00.377121 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:00.377121 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:00.377121 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:00.377121 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:00.377121 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:00.377121 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:00.377121 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:00.377121 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:00.377121 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:00.377121 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:00.377121 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:00.377121 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:00.377121 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:00.377121 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:00.377121 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:00.377121 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:00.377927 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:00.377927 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:00.377927 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:00.377927 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:00.377927 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:00.377927 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:00.377927 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:00.377927 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:00.377927 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:00.377927 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:00.377927 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 22:49:00.377927 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:00.380725 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 216/418] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:49:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:00.391295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8948990-60c4-4e8c-9232-f0f471f1baf3 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:00.510794 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8948990-60c4-4e8c-9232-f0f471f1baf3 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:00.556701 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035504c6-9866-4cee-a898-2d446e5e5027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:00.557378 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035504c6-9866-4cee-a898-2d446e5e5027 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:00.557658 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035504c6-9866-4cee-a898-2d446e5e5027 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:00.557932 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035504c6-9866-4cee-a898-2d446e5e5027 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:00.558701 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035504c6-9866-4cee-a898-2d446e5e5027 None admin] RBAC: Adding query filter params (name=member) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:00.559008 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035504c6-9866-4cee-a898-2d446e5e5027 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:00.560372 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035504c6-9866-4cee-a898-2d446e5e5027 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:00.569581 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 203/419] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:49:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:00.575577 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2572ed-4f49-4bd9-aa8d-ad07ae9073a6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:00.730086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2572ed-4f49-4bd9-aa8d-ad07ae9073a6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:00.778827 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f6dfe-a029-4830-b8a7-b996e5270173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:00.779493 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f6dfe-a029-4830-b8a7-b996e5270173 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:00.779777 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f6dfe-a029-4830-b8a7-b996e5270173 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:00.780057 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f6dfe-a029-4830-b8a7-b996e5270173 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:00.871207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f6dfe-a029-4830-b8a7-b996e5270173 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:00.872868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f6dfe-a029-4830-b8a7-b996e5270173 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:00.918535 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1926d8-6aae-4336-9e89-594636beaef2 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:00.919128 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1926d8-6aae-4336-9e89-594636beaef2 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:00.919622 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1926d8-6aae-4336-9e89-594636beaef2 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:00.921453 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 204/420] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:49:00.927531 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41f122-c150-4c0c-a6af-0c01f4b8ca0d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:00.928127 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41f122-c150-4c0c-a6af-0c01f4b8ca0d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:00.929105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41f122-c150-4c0c-a6af-0c01f4b8ca0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:00.953908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-bd41f122-c150-4c0c-a6af-0c01f4b8ca0d None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:00.961748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-bd41f122-c150-4c0c-a6af-0c01f4b8ca0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:01.005630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 217/421] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:49:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:49:01.011233 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f6dfe-a029-4830-b8a7-b996e5270173 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:01.096689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 205/422] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:49:01.117519 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f6dfe-a029-4830-b8a7-b996e5270173 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:01.145328 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41f122-c150-4c0c-a6af-0c01f4b8ca0d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:01.152078 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7db7e-27c6-465e-8dd0-23255b063043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:01.153163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7db7e-27c6-465e-8dd0-23255b063043 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:01.153594 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7db7e-27c6-465e-8dd0-23255b063043 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:01.154039 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7db7e-27c6-465e-8dd0-23255b063043 None admin] PATH_INFO: `/v3/projects/1a6770d749f644eea4fd1584b1503d28` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:01.155406 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7db7e-27c6-465e-8dd0-23255b063043 None admin] RBAC: Authorizing `identity:get_project(project_id=1a6770d749f644eea4fd1584b1503d28)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:01.157411 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7db7e-27c6-465e-8dd0-23255b063043 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:01.159310 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 218/423] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:49:01 2022] GET /identity/v3/projects/1a6770d749f644eea4fd1584b1503d28 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:01.166334 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7db7e-27c6-465e-8dd0-23255b063043 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:01.283612 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7db7e-27c6-465e-8dd0-23255b063043 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:01.325299 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df2f3d-fcee-472f-be10-5e24d6049df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:01.326171 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df2f3d-fcee-472f-be10-5e24d6049df0 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:01.326491 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df2f3d-fcee-472f-be10-5e24d6049df0 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:01.326765 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df2f3d-fcee-472f-be10-5e24d6049df0 None admin] PATH_INFO: `/v3/groups/d2102ce0e44a47a58cffd1e370f61bdd` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:01.328813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df2f3d-fcee-472f-be10-5e24d6049df0 None admin] RBAC: Authorizing `identity:get_group(group_id=d2102ce0e44a47a58cffd1e370f61bdd)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:01.330283 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df2f3d-fcee-472f-be10-5e24d6049df0 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:01.331912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 219/424] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 9 22:49:01 2022] GET /identity/v3/groups/d2102ce0e44a47a58cffd1e370f61bdd => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:01.343100 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df2f3d-fcee-472f-be10-5e24d6049df0 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:01.359910 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41f122-c150-4c0c-a6af-0c01f4b8ca0d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:01.404725 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7b62c-5989-4205-bff4-84443c842e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:01.405920 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7b62c-5989-4205-bff4-84443c842e08 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:01.406437 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7b62c-5989-4205-bff4-84443c842e08 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:01.406913 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7b62c-5989-4205-bff4-84443c842e08 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:01.418256 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7b62c-5989-4205-bff4-84443c842e08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:01.426816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-d2c7b62c-5989-4205-bff4-84443c842e08 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:49:01.426816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:01.426816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:01.426816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:01.426816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:01.426816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:01.426816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:01.426816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:01.426816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:01.426816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:01.426816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:01.426816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:01.426816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:01.426816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:01.426816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:01.426816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:01.427721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:01.427721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:01.427721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:01.427721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:01.427721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:01.427721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:01.427721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:01.427721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:01.427721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:01.427721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:01.427721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:01.427721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:01.427721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:01.427721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:01.427721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:01.427721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:01.429187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:01.429187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:01.429187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:01.429187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:01.429187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:01.429187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:01.429187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:01.429187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:01.429187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:01.429187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:01.429187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:49:01.429187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:01.431976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 206/425] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 9 22:49:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:01.433539 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e8d52-68b2-4e7a-abb5-a7ce232c4528 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:01.434027 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e8d52-68b2-4e7a-abb5-a7ce232c4528 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:01.434579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e8d52-68b2-4e7a-abb5-a7ce232c4528 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:01.436134 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 207/426] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:01.437216 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e8d52-68b2-4e7a-abb5-a7ce232c4528 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:01.465386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df2f3d-fcee-472f-be10-5e24d6049df0 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:01.499661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf2dbf-4789-4434-b107-0548545359ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:01.500627 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf2dbf-4789-4434-b107-0548545359ec None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:01.500893 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf2dbf-4789-4434-b107-0548545359ec None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:01.501164 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf2dbf-4789-4434-b107-0548545359ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:01.502294 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf2dbf-4789-4434-b107-0548545359ec None admin] RBAC: Adding query filter params (group.id=d2102ce0e44a47a58cffd1e370f61bdd, role.id=3631750e6bf34730aa3557cd90aa6108, scope.project.id=1a6770d749f644eea4fd1584b1503d28) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:01.502605 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf2dbf-4789-4434-b107-0548545359ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:01.504520 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf2dbf-4789-4434-b107-0548545359ec None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:01.514512 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 220/427] 10.222.0.23 () {56 vars in 1376 bytes} [Mon May 9 22:49:01 2022] GET /identity/v3/role_assignments?group.id=d2102ce0e44a47a58cffd1e370f61bdd&scope.project.id=1a6770d749f644eea4fd1584b1503d28&role.id=3631750e6bf34730aa3557cd90aa6108 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:01.515754 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa0147-826e-49de-b09d-af6b55d21682 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:01.516034 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa0147-826e-49de-b09d-af6b55d21682 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:01.516459 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa0147-826e-49de-b09d-af6b55d21682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:01.542282 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-1aaa0147-826e-49de-b09d-af6b55d21682 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:01.543846 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1aaa0147-826e-49de-b09d-af6b55d21682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:01.569340 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e8d52-68b2-4e7a-abb5-a7ce232c4528 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:01.647778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384fc3f1-ea61-4e48-981f-ff2eeb8c1893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:01.648517 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fc3f1-ea61-4e48-981f-ff2eeb8c1893 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:01.650411 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fc3f1-ea61-4e48-981f-ff2eeb8c1893 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:01.650747 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fc3f1-ea61-4e48-981f-ff2eeb8c1893 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:01.651510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384fc3f1-ea61-4e48-981f-ff2eeb8c1893 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:01.651812 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384fc3f1-ea61-4e48-981f-ff2eeb8c1893 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:01.657239 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384fc3f1-ea61-4e48-981f-ff2eeb8c1893 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:01.673477 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 208/428] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:01.679426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384fc3f1-ea61-4e48-981f-ff2eeb8c1893 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:01.688172 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 221/429] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:49:01.726213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa0147-826e-49de-b09d-af6b55d21682 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:01.797744 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384fc3f1-ea61-4e48-981f-ff2eeb8c1893 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:01.844102 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbef25-f237-412d-a1f0-9cda6204bd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:01.845508 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbef25-f237-412d-a1f0-9cda6204bd4e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:01.846045 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbef25-f237-412d-a1f0-9cda6204bd4e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:01.846497 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbef25-f237-412d-a1f0-9cda6204bd4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:01.851162 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa0147-826e-49de-b09d-af6b55d21682 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:01.899701 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b374b-0f5e-4bc9-b5fc-eb01fb8b4ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:01.900939 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b374b-0f5e-4bc9-b5fc-eb01fb8b4ed5 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:01.901316 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b374b-0f5e-4bc9-b5fc-eb01fb8b4ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:01.901688 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b374b-0f5e-4bc9-b5fc-eb01fb8b4ed5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:01.915324 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044b374b-0f5e-4bc9-b5fc-eb01fb8b4ed5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:01.925200 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-044b374b-0f5e-4bc9-b5fc-eb01fb8b4ed5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 22:49:01.925200 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:01.925200 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:01.925200 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:01.925200 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:01.925200 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:01.925200 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:01.925200 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:01.925200 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:01.925200 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:01.925200 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:01.925200 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:01.925200 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:01.925200 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:01.925200 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:01.925200 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:01.926143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:01.926143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:01.926143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:01.926143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:01.926143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:01.926143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:01.926143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:01.926143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:01.926143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:01.926143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:01.926143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:01.926143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:01.926143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:01.926143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:01.926143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:01.926143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:01.927267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:01.927267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:01.927267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:01.927267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:01.927267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:01.927267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:01.927267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:01.927267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:01.927267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:01.927267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:01.927267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 22:49:01.927267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:01.927267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 222/430] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:49:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:01.932941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b374b-0f5e-4bc9-b5fc-eb01fb8b4ed5 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:01.934110 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cbef25-f237-412d-a1f0-9cda6204bd4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:01.935822 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cbef25-f237-412d-a1f0-9cda6204bd4e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:02.048309 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b374b-0f5e-4bc9-b5fc-eb01fb8b4ed5 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:02.070154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 209/431] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:49:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 22:49:02.075140 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbef25-f237-412d-a1f0-9cda6204bd4e None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:02.098902 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4eb690-76f1-41a7-ad70-8e27d8790fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:02.099512 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4eb690-76f1-41a7-ad70-8e27d8790fe9 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:02.099828 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4eb690-76f1-41a7-ad70-8e27d8790fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:02.100104 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4eb690-76f1-41a7-ad70-8e27d8790fe9 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:02.100934 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4eb690-76f1-41a7-ad70-8e27d8790fe9 None admin] RBAC: Adding query filter params (name=member) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:02.101153 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4eb690-76f1-41a7-ad70-8e27d8790fe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:02.102702 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4eb690-76f1-41a7-ad70-8e27d8790fe9 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:02.112467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 223/432] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:49:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:02.123439 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4eb690-76f1-41a7-ad70-8e27d8790fe9 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:02.197879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbef25-f237-412d-a1f0-9cda6204bd4e None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:02.246648 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637aa28c-0e39-44c2-817d-554883b60f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:02.247285 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4eb690-76f1-41a7-ad70-8e27d8790fe9 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:02.248103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637aa28c-0e39-44c2-817d-554883b60f64 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:02.248731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637aa28c-0e39-44c2-817d-554883b60f64 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:02.249206 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637aa28c-0e39-44c2-817d-554883b60f64 None admin] PATH_INFO: `/v3/projects/1a6770d749f644eea4fd1584b1503d28` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:02.251381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637aa28c-0e39-44c2-817d-554883b60f64 None admin] RBAC: Authorizing `identity:get_project(project_id=1a6770d749f644eea4fd1584b1503d28)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:02.253104 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637aa28c-0e39-44c2-817d-554883b60f64 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:02.255309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 210/433] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:49:02 2022] GET /identity/v3/projects/1a6770d749f644eea4fd1584b1503d28 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:02.262156 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637aa28c-0e39-44c2-817d-554883b60f64 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:02.295629 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6fc33-ba35-43a6-a96e-a98b92958a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:02.296280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6fc33-ba35-43a6-a96e-a98b92958a9e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:02.296566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6fc33-ba35-43a6-a96e-a98b92958a9e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:02.296842 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6fc33-ba35-43a6-a96e-a98b92958a9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:02.387262 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6fc33-ba35-43a6-a96e-a98b92958a9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:02.389406 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6fc33-ba35-43a6-a96e-a98b92958a9e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:02.391717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637aa28c-0e39-44c2-817d-554883b60f64 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:02.428327 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf27ed-66d2-4027-8ea4-b6b101389d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:02.429307 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf27ed-66d2-4027-8ea4-b6b101389d91 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:02.429655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf27ed-66d2-4027-8ea4-b6b101389d91 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:02.429999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf27ed-66d2-4027-8ea4-b6b101389d91 None admin] PATH_INFO: `/v3/groups/d2102ce0e44a47a58cffd1e370f61bdd` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:02.431418 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf27ed-66d2-4027-8ea4-b6b101389d91 None admin] RBAC: Authorizing `identity:get_group(group_id=d2102ce0e44a47a58cffd1e370f61bdd)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:02.433045 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf27ed-66d2-4027-8ea4-b6b101389d91 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:02.435236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 211/434] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 9 22:49:02 2022] GET /identity/v3/groups/d2102ce0e44a47a58cffd1e370f61bdd => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:02.442696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf27ed-66d2-4027-8ea4-b6b101389d91 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:02.513298 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 224/435] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:49:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 22:49:02.519325 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6fc33-ba35-43a6-a96e-a98b92958a9e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:02.568659 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf27ed-66d2-4027-8ea4-b6b101389d91 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:02.606237 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c5d1c-13e7-4a57-ae6d-c4eaf754c11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:02.606868 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c5d1c-13e7-4a57-ae6d-c4eaf754c11e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:02.607148 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c5d1c-13e7-4a57-ae6d-c4eaf754c11e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:02.607422 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c5d1c-13e7-4a57-ae6d-c4eaf754c11e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:02.608255 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c5d1c-13e7-4a57-ae6d-c4eaf754c11e None admin] RBAC: Adding query filter params (group.id=d2102ce0e44a47a58cffd1e370f61bdd, role.id=46ddc3cf4b124399a275bc639635e10a, scope.project.id=1a6770d749f644eea4fd1584b1503d28) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:02.608568 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c5d1c-13e7-4a57-ae6d-c4eaf754c11e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:02.610306 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c5d1c-13e7-4a57-ae6d-c4eaf754c11e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:02.626687 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 212/436] 10.222.0.23 () {56 vars in 1376 bytes} [Mon May 9 22:49:02 2022] GET /identity/v3/role_assignments?group.id=d2102ce0e44a47a58cffd1e370f61bdd&scope.project.id=1a6770d749f644eea4fd1584b1503d28&role.id=46ddc3cf4b124399a275bc639635e10a => generated 611 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:02.628612 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61054670-3dee-47c4-bef3-20d602335aaf None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:02.628902 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61054670-3dee-47c4-bef3-20d602335aaf None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:02.629181 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61054670-3dee-47c4-bef3-20d602335aaf None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:02.630799 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 213/437] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:02.638233 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb4aea-fc11-4059-a602-9bbc42dd9f36 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:02.639601 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb4aea-fc11-4059-a602-9bbc42dd9f36 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:02.640698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb4aea-fc11-4059-a602-9bbc42dd9f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:02.644557 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6fc33-ba35-43a6-a96e-a98b92958a9e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:02.663714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e4eb4aea-fc11-4059-a602-9bbc42dd9f36 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:02.664973 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e4eb4aea-fc11-4059-a602-9bbc42dd9f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:02.686017 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a571d3-a347-4df1-bc4a-dd7fad77a219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:02.687312 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a571d3-a347-4df1-bc4a-dd7fad77a219 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:02.687805 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a571d3-a347-4df1-bc4a-dd7fad77a219 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:02.688283 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a571d3-a347-4df1-bc4a-dd7fad77a219 None admin] PATH_INFO: `/v3/projects/db7334eab91d41fa884c491d1364a253` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:02.689913 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a571d3-a347-4df1-bc4a-dd7fad77a219 None admin] RBAC: Authorizing `identity:get_project(project_id=db7334eab91d41fa884c491d1364a253)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:02.691714 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a571d3-a347-4df1-bc4a-dd7fad77a219 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:02.694024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 225/438] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:49:02 2022] GET /identity/v3/projects/db7334eab91d41fa884c491d1364a253 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:02.699579 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a571d3-a347-4df1-bc4a-dd7fad77a219 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:02.757942 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 214/439] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:49:02.781797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb4aea-fc11-4059-a602-9bbc42dd9f36 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:02.825113 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a571d3-a347-4df1-bc4a-dd7fad77a219 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:02.869624 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb6c66-266b-4345-9dcc-e813639caafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:02.870534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb6c66-266b-4345-9dcc-e813639caafe None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:02.871032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb6c66-266b-4345-9dcc-e813639caafe None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:02.871165 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb6c66-266b-4345-9dcc-e813639caafe None admin] PATH_INFO: `/v3/groups/d2102ce0e44a47a58cffd1e370f61bdd` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:02.872659 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb6c66-266b-4345-9dcc-e813639caafe None admin] RBAC: Authorizing `identity:get_group(group_id=d2102ce0e44a47a58cffd1e370f61bdd)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:02.874099 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb6c66-266b-4345-9dcc-e813639caafe None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:02.875985 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 226/440] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 9 22:49:02 2022] GET /identity/v3/groups/d2102ce0e44a47a58cffd1e370f61bdd => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:02.882087 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb6c66-266b-4345-9dcc-e813639caafe None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:02.910632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb4aea-fc11-4059-a602-9bbc42dd9f36 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:02.960781 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d2800-9ae3-43b5-987d-80e2de678b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:02.961605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d2800-9ae3-43b5-987d-80e2de678b71 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:02.963879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d2800-9ae3-43b5-987d-80e2de678b71 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:02.963879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d2800-9ae3-43b5-987d-80e2de678b71 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:02.972733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d2800-9ae3-43b5-987d-80e2de678b71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:02.980838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-b55d2800-9ae3-43b5-987d-80e2de678b71 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:49:02.980838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:02.980838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:02.980838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:02.980838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:02.980838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:02.980838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:02.980838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:02.980838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:02.980838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:02.980838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:02.980838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:02.980838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:02.980838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:02.980838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:02.980838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:02.981736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:02.981736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:02.981736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:02.981736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:02.981736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:02.981736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:02.981736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:02.981736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:02.981736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:02.981736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:02.981736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:02.981736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:02.981736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:02.981736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:02.981736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:02.981736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:02.982615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:02.982615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:02.982615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:02.982615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:02.982615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:02.982615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:02.982615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:02.982615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:02.982615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:02.982615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:02.982615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:49:02.982615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:02.982615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 215/441] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 9 22:49:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:02.983273 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a844e4-d715-4943-921f-3888400a0cf8 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:02.983833 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a844e4-d715-4943-921f-3888400a0cf8 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:02.983833 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a844e4-d715-4943-921f-3888400a0cf8 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:02.987283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 216/442] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:02.991765 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e2686-7900-4e24-85d8-de42a820ca22 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:02.992134 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e2686-7900-4e24-85d8-de42a820ca22 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:02.992544 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e2686-7900-4e24-85d8-de42a820ca22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:03.003831 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb6c66-266b-4345-9dcc-e813639caafe None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:03.014015 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c09e2686-7900-4e24-85d8-de42a820ca22 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:03.015298 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c09e2686-7900-4e24-85d8-de42a820ca22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:03.040492 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90acbf51-d211-4b36-a961-a73b0a3df256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:03.041143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90acbf51-d211-4b36-a961-a73b0a3df256 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:03.041423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90acbf51-d211-4b36-a961-a73b0a3df256 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:03.041700 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90acbf51-d211-4b36-a961-a73b0a3df256 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:03.042658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90acbf51-d211-4b36-a961-a73b0a3df256 None admin] RBAC: Adding query filter params (group.id=d2102ce0e44a47a58cffd1e370f61bdd, role.id=3631750e6bf34730aa3557cd90aa6108, scope.project.id=db7334eab91d41fa884c491d1364a253) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:03.042872 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90acbf51-d211-4b36-a961-a73b0a3df256 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:03.044298 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90acbf51-d211-4b36-a961-a73b0a3df256 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:03.050957 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 227/443] 10.222.0.23 () {56 vars in 1376 bytes} [Mon May 9 22:49:02 2022] GET /identity/v3/role_assignments?group.id=d2102ce0e44a47a58cffd1e370f61bdd&scope.project.id=db7334eab91d41fa884c491d1364a253&role.id=3631750e6bf34730aa3557cd90aa6108 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:03.055144 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90acbf51-d211-4b36-a961-a73b0a3df256 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:03.094365 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 217/444] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:49:03.119361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e2686-7900-4e24-85d8-de42a820ca22 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:03.170165 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90acbf51-d211-4b36-a961-a73b0a3df256 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:03.205014 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c57a73-e7ab-4104-9edb-6ace7ff0e552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:03.205703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c57a73-e7ab-4104-9edb-6ace7ff0e552 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:03.205967 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c57a73-e7ab-4104-9edb-6ace7ff0e552 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:03.206271 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c57a73-e7ab-4104-9edb-6ace7ff0e552 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:03.207032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c57a73-e7ab-4104-9edb-6ace7ff0e552 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:03.207332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c57a73-e7ab-4104-9edb-6ace7ff0e552 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:03.208710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c57a73-e7ab-4104-9edb-6ace7ff0e552 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:03.221891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 228/445] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:03.223040 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c57a73-e7ab-4104-9edb-6ace7ff0e552 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:03.226219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e2686-7900-4e24-85d8-de42a820ca22 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:03.267793 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d94966-d29c-4fd3-9912-895026a32c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:03.268570 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d94966-d29c-4fd3-9912-895026a32c0d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:03.268848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d94966-d29c-4fd3-9912-895026a32c0d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:03.269118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d94966-d29c-4fd3-9912-895026a32c0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:03.281340 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d94966-d29c-4fd3-9912-895026a32c0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:03.289821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-b9d94966-d29c-4fd3-9912-895026a32c0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:49:03.289821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:03.289821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:03.289821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:03.289821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:03.289821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:03.289821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:03.289821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:03.289821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:03.289821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:03.289821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:03.289821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:03.289821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:03.289821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:03.289821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:03.289821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:03.290895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:03.290895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:03.290895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:03.290895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:03.290895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:03.290895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:03.290895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:03.290895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:03.290895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:03.290895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:03.290895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:03.290895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:03.290895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:03.290895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:03.290895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:03.290895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:03.291904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:03.291904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:03.291904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:03.291904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:03.291904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:03.291904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:03.291904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:03.291904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:03.291904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:03.291904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:03.291904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:49:03.291904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:03.293749 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 218/446] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:49:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:03.299520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d94966-d29c-4fd3-9912-895026a32c0d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:03.339097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c57a73-e7ab-4104-9edb-6ace7ff0e552 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:03.375485 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1821a-049f-4ae2-920a-e7038ab0769e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:03.376163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1821a-049f-4ae2-920a-e7038ab0769e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:03.376442 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1821a-049f-4ae2-920a-e7038ab0769e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:03.376714 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1821a-049f-4ae2-920a-e7038ab0769e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:03.409127 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d94966-d29c-4fd3-9912-895026a32c0d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:03.445493 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990013c4-87e8-47f0-ab2a-347c31ba9c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:03.446079 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990013c4-87e8-47f0-ab2a-347c31ba9c6b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:03.446385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990013c4-87e8-47f0-ab2a-347c31ba9c6b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:03.446657 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990013c4-87e8-47f0-ab2a-347c31ba9c6b None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:03.447396 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990013c4-87e8-47f0-ab2a-347c31ba9c6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:03.447695 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990013c4-87e8-47f0-ab2a-347c31ba9c6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:03.449056 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990013c4-87e8-47f0-ab2a-347c31ba9c6b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:03.453089 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d1821a-049f-4ae2-920a-e7038ab0769e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:03.454575 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d1821a-049f-4ae2-920a-e7038ab0769e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:03.457111 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 219/447] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:49:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:03.463464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990013c4-87e8-47f0-ab2a-347c31ba9c6b None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:03.554820 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 229/448] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:49:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:49:03.560025 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990013c4-87e8-47f0-ab2a-347c31ba9c6b None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:03.560413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1821a-049f-4ae2-920a-e7038ab0769e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:03.591691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07ac92-913c-4b1d-8005-a2cf60031aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:03.592327 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07ac92-913c-4b1d-8005-a2cf60031aac None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:03.592602 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07ac92-913c-4b1d-8005-a2cf60031aac None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:03.592871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07ac92-913c-4b1d-8005-a2cf60031aac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:03.659128 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07ac92-913c-4b1d-8005-a2cf60031aac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:03.660594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07ac92-913c-4b1d-8005-a2cf60031aac None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:03.662979 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1821a-049f-4ae2-920a-e7038ab0769e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:03.694134 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70045e-1fe3-4092-9068-d364085248e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:03.694974 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70045e-1fe3-4092-9068-d364085248e8 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:03.695254 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70045e-1fe3-4092-9068-d364085248e8 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:03.695573 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70045e-1fe3-4092-9068-d364085248e8 None admin] PATH_INFO: `/v3/projects/db7334eab91d41fa884c491d1364a253` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:03.696693 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70045e-1fe3-4092-9068-d364085248e8 None admin] RBAC: Authorizing `identity:get_project(project_id=db7334eab91d41fa884c491d1364a253)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:03.698145 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70045e-1fe3-4092-9068-d364085248e8 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:03.699961 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 230/449] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:49:03 2022] GET /identity/v3/projects/db7334eab91d41fa884c491d1364a253 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:03.705298 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70045e-1fe3-4092-9068-d364085248e8 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:03.752783 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 220/450] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:49:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 22:49:03.758195 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07ac92-913c-4b1d-8005-a2cf60031aac None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:03.798109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70045e-1fe3-4092-9068-d364085248e8 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:03.829806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bafc3b-0544-48ee-996e-f04a5c54bc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:03.830668 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bafc3b-0544-48ee-996e-f04a5c54bc1f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:03.830944 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bafc3b-0544-48ee-996e-f04a5c54bc1f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:03.831212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bafc3b-0544-48ee-996e-f04a5c54bc1f None admin] PATH_INFO: `/v3/groups/d2102ce0e44a47a58cffd1e370f61bdd` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:03.832382 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bafc3b-0544-48ee-996e-f04a5c54bc1f None admin] RBAC: Authorizing `identity:get_group(group_id=d2102ce0e44a47a58cffd1e370f61bdd)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:03.833794 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bafc3b-0544-48ee-996e-f04a5c54bc1f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:03.835980 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 231/451] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 9 22:49:03 2022] GET /identity/v3/groups/d2102ce0e44a47a58cffd1e370f61bdd => generated 226 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:03.841631 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bafc3b-0544-48ee-996e-f04a5c54bc1f None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:03.848215 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07ac92-913c-4b1d-8005-a2cf60031aac None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:03.886218 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1aa45-455c-42df-858a-3797851aa977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:03.887068 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1aa45-455c-42df-858a-3797851aa977 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:03.887345 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1aa45-455c-42df-858a-3797851aa977 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:03.887616 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1aa45-455c-42df-858a-3797851aa977 None admin] PATH_INFO: `/v3/projects/4202d89a17374bc0a3d992386d254d4c` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:03.888790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af1aa45-455c-42df-858a-3797851aa977 None admin] RBAC: Authorizing `identity:get_project(project_id=4202d89a17374bc0a3d992386d254d4c)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:03.890266 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af1aa45-455c-42df-858a-3797851aa977 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:03.891953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 221/452] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:49:03 2022] GET /identity/v3/projects/4202d89a17374bc0a3d992386d254d4c => generated 344 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:03.897831 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1aa45-455c-42df-858a-3797851aa977 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:03.939191 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bafc3b-0544-48ee-996e-f04a5c54bc1f None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:03.971440 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b115c8-bd13-4de5-bfc4-826862791a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:03.972017 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b115c8-bd13-4de5-bfc4-826862791a61 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:03.972300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b115c8-bd13-4de5-bfc4-826862791a61 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:03.972630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b115c8-bd13-4de5-bfc4-826862791a61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:03.973461 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b115c8-bd13-4de5-bfc4-826862791a61 None admin] RBAC: Adding query filter params (group.id=d2102ce0e44a47a58cffd1e370f61bdd, role.id=46ddc3cf4b124399a275bc639635e10a, scope.project.id=db7334eab91d41fa884c491d1364a253) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:03.973772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b115c8-bd13-4de5-bfc4-826862791a61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:03.975262 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b115c8-bd13-4de5-bfc4-826862791a61 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:03.980836 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 232/453] 10.222.0.23 () {56 vars in 1376 bytes} [Mon May 9 22:49:03 2022] GET /identity/v3/role_assignments?group.id=d2102ce0e44a47a58cffd1e370f61bdd&scope.project.id=db7334eab91d41fa884c491d1364a253&role.id=46ddc3cf4b124399a275bc639635e10a => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:03.988641 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1aa45-455c-42df-858a-3797851aa977 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:04.024499 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a4b41-39fc-4e29-8a3f-1555f4d0f364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:04.025360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a4b41-39fc-4e29-8a3f-1555f4d0f364 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:04.025634 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a4b41-39fc-4e29-8a3f-1555f4d0f364 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:04.025898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a4b41-39fc-4e29-8a3f-1555f4d0f364 None admin] PATH_INFO: `/v3/groups/57bf5890247b4a7995c02be731a983ce` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:04.027099 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a4b41-39fc-4e29-8a3f-1555f4d0f364 None admin] RBAC: Authorizing `identity:get_group(group_id=57bf5890247b4a7995c02be731a983ce)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:04.028543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a4b41-39fc-4e29-8a3f-1555f4d0f364 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:04.030202 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 222/454] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 9 22:49:03 2022] GET /identity/v3/groups/57bf5890247b4a7995c02be731a983ce => generated 229 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:04.036190 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b115c8-bd13-4de5-bfc4-826862791a61 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:04.140352 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b115c8-bd13-4de5-bfc4-826862791a61 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:04.174186 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9590b4-667c-4eaf-bcdd-6391d6f9b1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:04.174839 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9590b4-667c-4eaf-bcdd-6391d6f9b1d7 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:04.175117 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9590b4-667c-4eaf-bcdd-6391d6f9b1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:04.176374 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9590b4-667c-4eaf-bcdd-6391d6f9b1d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:04.177229 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9590b4-667c-4eaf-bcdd-6391d6f9b1d7 None admin] RBAC: Adding query filter params (group.id=57bf5890247b4a7995c02be731a983ce, role.id=bda550492f4f4c5c83f52fa89fc18a7d, scope.project.id=4202d89a17374bc0a3d992386d254d4c) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:04.177534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9590b4-667c-4eaf-bcdd-6391d6f9b1d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:04.179685 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9590b4-667c-4eaf-bcdd-6391d6f9b1d7 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:04.185801 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 233/455] 10.222.0.23 () {56 vars in 1376 bytes} [Mon May 9 22:49:04 2022] GET /identity/v3/role_assignments?group.id=57bf5890247b4a7995c02be731a983ce&scope.project.id=4202d89a17374bc0a3d992386d254d4c&role.id=bda550492f4f4c5c83f52fa89fc18a7d => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:06.854927 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2558e-a24a-47dd-88ae-914466b8f607 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:06.855941 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2558e-a24a-47dd-88ae-914466b8f607 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:06.856438 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2558e-a24a-47dd-88ae-914466b8f607 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:06.858386 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 223/456] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:06.863849 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040e760-f22c-4078-9402-c62613d16b01 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:06.864283 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040e760-f22c-4078-9402-c62613d16b01 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:06.864459 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040e760-f22c-4078-9402-c62613d16b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:06.888136 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-9040e760-f22c-4078-9402-c62613d16b01 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:06.889247 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9040e760-f22c-4078-9402-c62613d16b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:06.979908 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 234/457] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:49:07.016928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2558e-a24a-47dd-88ae-914466b8f607 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:07.153811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2558e-a24a-47dd-88ae-914466b8f607 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:07.180918 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b71dc5-9eaf-4c2d-b9d8-891b412f9ba7 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:07.181285 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b71dc5-9eaf-4c2d-b9d8-891b412f9ba7 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:07.181477 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b71dc5-9eaf-4c2d-b9d8-891b412f9ba7 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:07.186198 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 235/458] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:07.195513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce6a09-eee8-4215-a6ef-51a432f65dd8 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:07.195798 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce6a09-eee8-4215-a6ef-51a432f65dd8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:07.196078 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce6a09-eee8-4215-a6ef-51a432f65dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:07.197319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423e332-0431-47de-abdf-844a4c555a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:07.198235 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423e332-0431-47de-abdf-844a4c555a11 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:07.198518 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423e332-0431-47de-abdf-844a4c555a11 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:07.198786 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423e332-0431-47de-abdf-844a4c555a11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:07.215651 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423e332-0431-47de-abdf-844a4c555a11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:07.217112 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423e332-0431-47de-abdf-844a4c555a11 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:07.230901 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-04ce6a09-eee8-4215-a6ef-51a432f65dd8 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:07.233890 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-04ce6a09-eee8-4215-a6ef-51a432f65dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:07.247655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-e423e332-0431-47de-abdf-844a4c555a11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:07.247655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:07.247655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:07.247655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:07.247655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:07.247655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:07.247655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:07.247655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:07.247655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:07.247655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:07.247655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:07.247655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:07.247655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:07.247655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:07.247655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:07.248594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:07.248594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:07.248594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:07.248594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:07.248594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:07.248594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:07.248594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:07.248594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:07.248594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:07.248594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:07.248594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:07.248594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:07.248594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:07.248594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:07.248594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:07.248594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:07.248594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:07.249462 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:07.249462 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:07.249462 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:07.249462 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:07.256395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 224/459] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:07.257284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423e332-0431-47de-abdf-844a4c555a11 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:07.324062 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 236/460] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:49:07.325467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b985f5-08a6-4c8b-95e1-d0c96b48afbc None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:07.325750 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b985f5-08a6-4c8b-95e1-d0c96b48afbc None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:07.326021 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b985f5-08a6-4c8b-95e1-d0c96b48afbc None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:07.327579 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 237/461] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:07.339808 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64624e8d-b747-4700-a8a0-0bfe23d2489f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:07.340302 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64624e8d-b747-4700-a8a0-0bfe23d2489f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:07.340748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64624e8d-b747-4700-a8a0-0bfe23d2489f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:07.368039 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-64624e8d-b747-4700-a8a0-0bfe23d2489f None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:07.369835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-64624e8d-b747-4700-a8a0-0bfe23d2489f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:07.382193 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423e332-0431-47de-abdf-844a4c555a11 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:07.423020 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff8750-9271-4b8c-869f-8897fbe0e2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:07.424464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff8750-9271-4b8c-869f-8897fbe0e2b5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:07.424893 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff8750-9271-4b8c-869f-8897fbe0e2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:07.425309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff8750-9271-4b8c-869f-8897fbe0e2b5 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:07.426380 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffff8750-9271-4b8c-869f-8897fbe0e2b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:07.427230 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffff8750-9271-4b8c-869f-8897fbe0e2b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:07.429590 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffff8750-9271-4b8c-869f-8897fbe0e2b5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:07.449755 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 225/462] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:07.450912 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff8750-9271-4b8c-869f-8897fbe0e2b5 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:07.453421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 238/463] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:49:07.454562 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64624e8d-b747-4700-a8a0-0bfe23d2489f None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:07.562850 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff8750-9271-4b8c-869f-8897fbe0e2b5 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:07.565025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64624e8d-b747-4700-a8a0-0bfe23d2489f None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:07.595426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cabbae-9a2b-40a3-aea6-45b96f95bfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:07.596210 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cabbae-9a2b-40a3-aea6-45b96f95bfa1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:07.596490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cabbae-9a2b-40a3-aea6-45b96f95bfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:07.596763 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cabbae-9a2b-40a3-aea6-45b96f95bfa1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:07.598587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c100b-aeca-45f1-92d7-da74d38f7ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:07.599243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c100b-aeca-45f1-92d7-da74d38f7ea2 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:07.600087 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c100b-aeca-45f1-92d7-da74d38f7ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:07.600333 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c100b-aeca-45f1-92d7-da74d38f7ea2 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:07.601028 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c100b-aeca-45f1-92d7-da74d38f7ea2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:07.602474 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c100b-aeca-45f1-92d7-da74d38f7ea2 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:07.607567 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cabbae-9a2b-40a3-aea6-45b96f95bfa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:07.608994 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cabbae-9a2b-40a3-aea6-45b96f95bfa1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:07.627264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-74cabbae-9a2b-40a3-aea6-45b96f95bfa1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:07.627264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:07.627264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:07.627264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:07.627264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:07.627264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:07.627264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:07.627264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:07.627264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:07.627264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:07.627264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:07.627264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:07.627264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:07.627264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:07.627264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:07.628347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:07.628347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:07.628347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:07.628347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:07.628347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:07.628347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:07.628347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:07.628347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:07.628347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:07.628347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:07.628347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:07.628347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:07.628347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:07.628347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:07.628347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:07.628347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:07.628347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:07.629430 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:07.629430 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:07.629430 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:07.629430 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:07.630013 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 239/464] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 9 22:49:07 2022] POST /identity/v3/users => generated 246 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:49:07.630013 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c100b-aeca-45f1-92d7-da74d38f7ea2 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:07.633815 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 226/465] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:07.642661 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24c437-dc3e-4c0e-944e-d899f04817d8 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:07.642661 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24c437-dc3e-4c0e-944e-d899f04817d8 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:07.642661 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24c437-dc3e-4c0e-944e-d899f04817d8 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:07.642661 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 227/466] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:07.642661 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24c437-dc3e-4c0e-944e-d899f04817d8 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:07.766176 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c100b-aeca-45f1-92d7-da74d38f7ea2 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:07.769013 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24c437-dc3e-4c0e-944e-d899f04817d8 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:07.807288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f28bd6-c87c-4008-8e73-0a208b14e0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:07.807560 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85144eb3-6da5-41aa-8339-92e675493b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:07.808191 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f28bd6-c87c-4008-8e73-0a208b14e0ee None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:07.808613 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f28bd6-c87c-4008-8e73-0a208b14e0ee None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:07.809414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f28bd6-c87c-4008-8e73-0a208b14e0ee None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:07.809695 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85144eb3-6da5-41aa-8339-92e675493b5f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:07.809998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f28bd6-c87c-4008-8e73-0a208b14e0ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:07.810332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85144eb3-6da5-41aa-8339-92e675493b5f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:07.810556 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f28bd6-c87c-4008-8e73-0a208b14e0ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:07.811091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85144eb3-6da5-41aa-8339-92e675493b5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:07.812751 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f28bd6-c87c-4008-8e73-0a208b14e0ee None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:07.832283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 228/467] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:07.832283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec9094-5904-4367-a747-cb04aecb659d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:07.832283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec9094-5904-4367-a747-cb04aecb659d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:07.832283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec9094-5904-4367-a747-cb04aecb659d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:07.834447 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85144eb3-6da5-41aa-8339-92e675493b5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:07.836399 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85144eb3-6da5-41aa-8339-92e675493b5f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:07.852503 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-cdec9094-5904-4367-a747-cb04aecb659d None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:07.852994 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-cdec9094-5904-4367-a747-cb04aecb659d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:07.859290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-85144eb3-6da5-41aa-8339-92e675493b5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:07.859290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:07.859290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:07.859290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:07.859290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:07.859290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:07.859290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:07.859290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:07.859290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:07.859290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:07.859290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:07.859290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:07.859290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:07.859290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:07.859290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:07.860509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:07.860509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:07.860509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:07.860509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:07.860509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:07.860509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:07.860509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:07.860509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:07.860509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:07.860509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:07.860509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:07.860509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:07.860509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:07.860509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:07.860509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:07.860509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:07.860509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:07.861344 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:07.861344 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:07.861344 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:07.861344 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:07.863918 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 240/468] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:07.865900 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85144eb3-6da5-41aa-8339-92e675493b5f None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:07.930469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 229/469] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:49:07.932037 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec9094-5904-4367-a747-cb04aecb659d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:07.978143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85144eb3-6da5-41aa-8339-92e675493b5f None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:08.006944 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e4445-213e-45e3-8ab4-181ecdc7f8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:08.007668 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e4445-213e-45e3-8ab4-181ecdc7f8f0 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:08.007953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e4445-213e-45e3-8ab4-181ecdc7f8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:08.008225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e4445-213e-45e3-8ab4-181ecdc7f8f0 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:08.008923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e4445-213e-45e3-8ab4-181ecdc7f8f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:08.010384 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e4445-213e-45e3-8ab4-181ecdc7f8f0 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:08.033687 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 241/470] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 9 22:49:07 2022] POST /identity/v3/users => generated 248 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:49:08.034820 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e4445-213e-45e3-8ab4-181ecdc7f8f0 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:08.096934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec9094-5904-4367-a747-cb04aecb659d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:08.165866 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e4445-213e-45e3-8ab4-181ecdc7f8f0 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:08.173169 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfed674-74b7-4139-a0c9-870863e0dad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:08.174849 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfed674-74b7-4139-a0c9-870863e0dad4 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:08.175343 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfed674-74b7-4139-a0c9-870863e0dad4 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:08.175815 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfed674-74b7-4139-a0c9-870863e0dad4 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:08.176766 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfed674-74b7-4139-a0c9-870863e0dad4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:08.177291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfed674-74b7-4139-a0c9-870863e0dad4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:08.179014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfed674-74b7-4139-a0c9-870863e0dad4 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:08.203955 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43878f-8133-430d-a487-f567e51c4ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:08.204572 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 230/471] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:08.204710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43878f-8133-430d-a487-f567e51c4ec0 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:08.204966 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43878f-8133-430d-a487-f567e51c4ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:08.206265 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43878f-8133-430d-a487-f567e51c4ec0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:08.211431 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfed674-74b7-4139-a0c9-870863e0dad4 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:08.218864 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce43878f-8133-430d-a487-f567e51c4ec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:08.220426 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce43878f-8133-430d-a487-f567e51c4ec0 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:08.241999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-ce43878f-8133-430d-a487-f567e51c4ec0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:08.241999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:08.241999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:08.241999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:08.241999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:08.241999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:08.241999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:08.241999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:08.241999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:08.241999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:08.241999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:08.241999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:08.241999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:08.241999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:08.241999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:08.242936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:08.242936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:08.242936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:08.242936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:08.242936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:08.242936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:08.242936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:08.242936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:08.242936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:08.242936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:08.242936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:08.242936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:08.242936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:08.242936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:08.242936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:08.242936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:08.242936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:08.243828 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:08.243828 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:08.243828 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:08.243828 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:08.243828 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 242/472] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:08.249611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43878f-8133-430d-a487-f567e51c4ec0 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:08.370684 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfed674-74b7-4139-a0c9-870863e0dad4 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:08.385121 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43878f-8133-430d-a487-f567e51c4ec0 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:08.414070 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c035523-716d-4a65-bb4f-7f26fd67929b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:08.415109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c035523-716d-4a65-bb4f-7f26fd67929b None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:08.415609 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c035523-716d-4a65-bb4f-7f26fd67929b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:08.416074 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c035523-716d-4a65-bb4f-7f26fd67929b None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:08.416985 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c035523-716d-4a65-bb4f-7f26fd67929b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:08.418648 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c035523-716d-4a65-bb4f-7f26fd67929b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:08.423856 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436d585-6e07-4493-960c-cc5a87c7f30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:08.424507 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436d585-6e07-4493-960c-cc5a87c7f30d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:08.424781 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436d585-6e07-4493-960c-cc5a87c7f30d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:08.425048 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436d585-6e07-4493-960c-cc5a87c7f30d None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:08.425757 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2436d585-6e07-4493-960c-cc5a87c7f30d None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:08.426053 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2436d585-6e07-4493-960c-cc5a87c7f30d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:08.427744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2436d585-6e07-4493-960c-cc5a87c7f30d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:08.444343 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 243/473] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:08.446798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 231/474] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 9 22:49:08 2022] POST /identity/v3/users => generated 248 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 22:49:08.449662 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436d585-6e07-4493-960c-cc5a87c7f30d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:08.499918 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42386268-1942-4c8b-9485-3bc57f6b1d0b None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:08.500228 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42386268-1942-4c8b-9485-3bc57f6b1d0b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:08.500479 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42386268-1942-4c8b-9485-3bc57f6b1d0b None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:08.508608 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 232/475] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:08 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:49:08.510078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce28c6-eefa-4a0a-99e3-a3c2db34a902 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:08.510409 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce28c6-eefa-4a0a-99e3-a3c2db34a902 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:08.510686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce28c6-eefa-4a0a-99e3-a3c2db34a902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:08.539062 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-49ce28c6-eefa-4a0a-99e3-a3c2db34a902 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:08.540274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-49ce28c6-eefa-4a0a-99e3-a3c2db34a902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:08.599555 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436d585-6e07-4493-960c-cc5a87c7f30d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:08.612476 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 233/476] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:49:08.638300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce28c6-eefa-4a0a-99e3-a3c2db34a902 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:08.638494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1a834-6537-41e4-8f5b-e347f7adb082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:08.638980 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1a834-6537-41e4-8f5b-e347f7adb082 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:08.639254 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1a834-6537-41e4-8f5b-e347f7adb082 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:08.639525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1a834-6537-41e4-8f5b-e347f7adb082 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:08.640485 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf1a834-6537-41e4-8f5b-e347f7adb082 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:08.644141 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf1a834-6537-41e4-8f5b-e347f7adb082 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:08.670684 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 244/477] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 9 22:49:08 2022] POST /identity/v3/users => generated 249 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:49:08.757435 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce28c6-eefa-4a0a-99e3-a3c2db34a902 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:08.794211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3dd254-d36d-40df-a184-1e3a3063ccfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:08.794889 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3dd254-d36d-40df-a184-1e3a3063ccfa None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:08.795166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3dd254-d36d-40df-a184-1e3a3063ccfa None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:08.795437 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3dd254-d36d-40df-a184-1e3a3063ccfa None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:08.796181 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3dd254-d36d-40df-a184-1e3a3063ccfa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:08.797596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3dd254-d36d-40df-a184-1e3a3063ccfa None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:08.806054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.sql.core [None req-5c3dd254-d36d-40df-a184-1e3a3063ccfa None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 09 22:49:08.806054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 09 22:49:08.806054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [parameters: {'id': 'ebb4449fc1634654829f500476e1deda', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 09 22:49:08.806054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=79888) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 09 22:49:08.829537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-5c3dd254-d36d-40df-a184-1e3a3063ccfa None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 09 22:49:08.829537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:08.829537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 22:49:08.829537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 22:49:08.829537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 22:49:08.829537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 22:49:08.829537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 22:49:08.829537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application result = self._query(query) May 09 22:49:08.829537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 22:49:08.829537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application conn.query(q) May 09 22:49:08.829537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 22:49:08.829537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 22:49:08.829537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 22:49:08.829537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application result.read() May 09 22:49:08.829537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 22:49:08.830500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 22:49:08.830500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 22:49:08.830500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 22:49:08.830500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 22:49:08.830500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 22:49:08.830500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 22:49:08.830500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 22:49:08.830500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 09 22:49:08.830500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:08.830500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 09 22:49:08.830500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:08.830500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:08.830500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 09 22:49:08.830500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 09 22:49:08.830500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 09 22:49:08.830500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return ref.to_dict() May 09 22:49:08.830500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 22:49:08.830500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application next(self.gen) May 09 22:49:08.831404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 09 22:49:08.831404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application yield resource May 09 22:49:08.831404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 22:49:08.831404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application next(self.gen) May 09 22:49:08.831404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 09 22:49:08.831404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.session.rollback() May 09 22:49:08.831404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 09 22:49:08.831404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.force_reraise() May 09 22:49:08.831404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 09 22:49:08.831404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise self.value May 09 22:49:08.831404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 09 22:49:08.831404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 09 22:49:08.831404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 09 22:49:08.831404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application session.commit() May 09 22:49:08.831404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 09 22:49:08.831404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 09 22:49:08.831404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 09 22:49:08.832259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._prepare_impl() May 09 22:49:08.832259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 09 22:49:08.832259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.session.flush() May 09 22:49:08.832259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 09 22:49:08.832259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._flush(objects) May 09 22:49:08.832259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 09 22:49:08.832259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 09 22:49:08.832259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 09 22:49:08.832259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application compat.raise_( May 09 22:49:08.832259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 22:49:08.832259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception May 09 22:49:08.832259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 09 22:49:08.832259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application flush_context.execute() May 09 22:49:08.832259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 09 22:49:08.832259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rec.execute(self) May 09 22:49:08.832259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 09 22:49:08.832259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 09 22:49:08.834510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 09 22:49:08.834510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application _emit_insert_statements( May 09 22:49:08.834510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 09 22:49:08.834510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application c = connection._execute_20( May 09 22:49:08.834510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 09 22:49:08.834510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 09 22:49:08.834510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 09 22:49:08.834510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 09 22:49:08.834510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 09 22:49:08.834510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ret = self._execute_context( May 09 22:49:08.834510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 09 22:49:08.834510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 09 22:49:08.834510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 09 22:49:08.834510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 09 22:49:08.834510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 22:49:08.834510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception May 09 22:49:08.836442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 22:49:08.836442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 22:49:08.836442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 22:49:08.836442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 22:49:08.836442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 22:49:08.836442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application result = self._query(query) May 09 22:49:08.836442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 22:49:08.836442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application conn.query(q) May 09 22:49:08.836442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 22:49:08.836442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 22:49:08.836442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 22:49:08.836442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application result.read() May 09 22:49:08.836442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 22:49:08.836442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 22:49:08.836442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 22:49:08.836442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 22:49:08.838676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 22:49:08.838676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 22:49:08.838676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 22:49:08.838676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 22:49:08.838676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 09 22:49:08.838676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 09 22:49:08.838676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [parameters: {'id': 'ebb4449fc1634654829f500476e1deda', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 09 22:49:08.838676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 09 22:49:08.838676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:08.838676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:49:08.838676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:08.838676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:08.838676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:08.838676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:08.838676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:08.838676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:08.840640 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:08.840640 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:08.840640 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:08.840640 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:08.840640 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:08.840640 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:08.840640 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 09 22:49:08.840640 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 09 22:49:08.840640 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:08.840640 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:08.840640 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 09 22:49:08.840640 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 09 22:49:08.840640 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 09 22:49:08.840640 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 09 22:49:08.840640 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 09 22:49:08.840640 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:08.842803 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 234/478] 10.222.0.23 () {60 vars in 1131 bytes} [Mon May 9 22:49:08 2022] POST /identity/v3/roles => generated 144 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:08.842940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1a834-6537-41e4-8f5b-e347f7adb082 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:08.949617 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1a834-6537-41e4-8f5b-e347f7adb082 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:08.990829 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de6c1a-3c59-43d3-a52c-e12b1b80d819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:08.991739 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de6c1a-3c59-43d3-a52c-e12b1b80d819 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:08.992015 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de6c1a-3c59-43d3-a52c-e12b1b80d819 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:08.992334 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de6c1a-3c59-43d3-a52c-e12b1b80d819 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:09.002519 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de6c1a-3c59-43d3-a52c-e12b1b80d819 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:09.011215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-59de6c1a-3c59-43d3-a52c-e12b1b80d819 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:49:09.011215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:09.011215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:09.011215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:09.011215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:09.011215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:09.011215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:09.011215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:09.011215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:09.011215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:09.011215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:09.011215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:09.011215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:09.011215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:09.011215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:09.011215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:09.013275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:09.013275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:09.013275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:09.013275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:09.013275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:09.013275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:09.013275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:09.013275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:09.013275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:09.013275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:09.013275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:09.013275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:09.013275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:09.013275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:09.013275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:09.013275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:09.015221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:09.015221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:09.015221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:09.015221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:09.015221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:09.015221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:09.015221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:09.015221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:09.015221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:09.015221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:09.015221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 22:49:09.015221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:09.015221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 245/479] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 9 22:49:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:09.018018 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3dd254-d36d-40df-a184-1e3a3063ccfa None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:09.140126 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3dd254-d36d-40df-a184-1e3a3063ccfa None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:09.177612 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8057e5-1a7e-4b3f-8a20-3016d7cb1648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:09.178483 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8057e5-1a7e-4b3f-8a20-3016d7cb1648 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:09.178796 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8057e5-1a7e-4b3f-8a20-3016d7cb1648 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:09.179071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8057e5-1a7e-4b3f-8a20-3016d7cb1648 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:09.179824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8057e5-1a7e-4b3f-8a20-3016d7cb1648 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:09.180128 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8057e5-1a7e-4b3f-8a20-3016d7cb1648 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:09.181490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8057e5-1a7e-4b3f-8a20-3016d7cb1648 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:09.190055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 235/480] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:10.119623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ead682-51fc-490c-ac92-36f3b14e921c None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:10.120319 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ead682-51fc-490c-ac92-36f3b14e921c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:10.120319 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ead682-51fc-490c-ac92-36f3b14e921c None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:10.121804 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 246/481] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:10.126733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a3628-fd43-40c5-8013-141690fc8179 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:10.127239 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a3628-fd43-40c5-8013-141690fc8179 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:10.127413 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a3628-fd43-40c5-8013-141690fc8179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:10.147125 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-0b0a3628-fd43-40c5-8013-141690fc8179 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:10.148291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0b0a3628-fd43-40c5-8013-141690fc8179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:10.227528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 236/482] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:49:10.262431 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ead682-51fc-490c-ac92-36f3b14e921c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:10.373665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ead682-51fc-490c-ac92-36f3b14e921c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:10.414160 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc533ea-ecbf-41b0-9a4a-87b2c949a385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:10.415012 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc533ea-ecbf-41b0-9a4a-87b2c949a385 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:10.415283 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc533ea-ecbf-41b0-9a4a-87b2c949a385 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:10.415548 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc533ea-ecbf-41b0-9a4a-87b2c949a385 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:10.429757 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc533ea-ecbf-41b0-9a4a-87b2c949a385 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:10.437449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-dbc533ea-ecbf-41b0-9a4a-87b2c949a385 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:10.437449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:10.437449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:10.437449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:10.437449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:10.437449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:10.437449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:10.437449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:10.437449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:10.437449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:10.437449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:10.437449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:10.437449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:10.437449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:10.437449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:10.437449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:10.439532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:10.439532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:10.439532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:10.439532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:10.439532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:10.439532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:10.439532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:10.439532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:10.439532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:10.439532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:10.439532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:10.439532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:10.439532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:10.439532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:10.439532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:10.439532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:10.441580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:10.441580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:10.441580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:10.441580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:10.441580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:10.441580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:10.441580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:10.441580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:10.441580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:10.441580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:10.441580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:10.441580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:10.443139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 247/483] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 9 22:49:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:10.446151 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a3628-fd43-40c5-8013-141690fc8179 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:10.572246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a3628-fd43-40c5-8013-141690fc8179 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:10.604889 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b7141-67cc-4bbe-b693-213211b43790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:10.605542 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b7141-67cc-4bbe-b693-213211b43790 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:10.605819 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b7141-67cc-4bbe-b693-213211b43790 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:10.606090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b7141-67cc-4bbe-b693-213211b43790 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:10.606976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b7141-67cc-4bbe-b693-213211b43790 None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:10.607279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b7141-67cc-4bbe-b693-213211b43790 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:10.608733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b7141-67cc-4bbe-b693-213211b43790 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:10.625875 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 237/484] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 9 22:49:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:10.630220 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc533ea-ecbf-41b0-9a4a-87b2c949a385 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:10.744612 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc533ea-ecbf-41b0-9a4a-87b2c949a385 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:10.776428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2573cc-cddf-4f2b-b89f-996db8fa9fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:10.777207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2573cc-cddf-4f2b-b89f-996db8fa9fd6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:10.777478 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2573cc-cddf-4f2b-b89f-996db8fa9fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:10.777742 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2573cc-cddf-4f2b-b89f-996db8fa9fd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:10.789525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2573cc-cddf-4f2b-b89f-996db8fa9fd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:10.791002 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2573cc-cddf-4f2b-b89f-996db8fa9fd6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:10.808898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-2d2573cc-cddf-4f2b-b89f-996db8fa9fd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:10.808898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:10.808898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:10.808898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:10.808898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:10.808898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:10.808898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:10.808898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:10.808898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:10.808898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:10.808898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:10.808898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:10.808898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:10.808898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:10.808898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:10.811008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:10.811008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:10.811008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:10.811008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:10.811008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:10.811008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:10.811008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:10.811008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:10.811008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:10.811008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:10.811008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:10.811008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:10.811008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:10.811008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:10.811008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:10.811008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:10.811008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:10.813055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:10.813055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:10.813055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:10.813055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:10.813055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 248/485] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:10.815651 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b7141-67cc-4bbe-b693-213211b43790 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:10.968175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b7141-67cc-4bbe-b693-213211b43790 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:11.017457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55681da-1292-4075-8a3f-35fd33ad60bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:11.018145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55681da-1292-4075-8a3f-35fd33ad60bb None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:11.018439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55681da-1292-4075-8a3f-35fd33ad60bb None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:11.018710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55681da-1292-4075-8a3f-35fd33ad60bb None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:11.019427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55681da-1292-4075-8a3f-35fd33ad60bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:11.019730 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55681da-1292-4075-8a3f-35fd33ad60bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:11.021088 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55681da-1292-4075-8a3f-35fd33ad60bb None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:11.040619 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 238/486] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:11.045847 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2573cc-cddf-4f2b-b89f-996db8fa9fd6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:11.055261 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ae763-38c4-437d-83ab-0a710172e3ee None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:11.055572 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ae763-38c4-437d-83ab-0a710172e3ee None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:11.055832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ae763-38c4-437d-83ab-0a710172e3ee None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:11.057599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 239/487] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:49:11.065337 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d62308-2386-4dc9-8a41-6f8da5f6fb10 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:11.066051 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d62308-2386-4dc9-8a41-6f8da5f6fb10 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:11.066374 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d62308-2386-4dc9-8a41-6f8da5f6fb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:11.090061 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-d4d62308-2386-4dc9-8a41-6f8da5f6fb10 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:11.090061 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d4d62308-2386-4dc9-8a41-6f8da5f6fb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:11.174465 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2573cc-cddf-4f2b-b89f-996db8fa9fd6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:11.232324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 240/488] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:49:11.238879 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbedbd2e-3252-4937-8582-df0659b44994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:11.239680 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbedbd2e-3252-4937-8582-df0659b44994 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:11.239988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbedbd2e-3252-4937-8582-df0659b44994 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:11.240262 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbedbd2e-3252-4937-8582-df0659b44994 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:11.248854 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbedbd2e-3252-4937-8582-df0659b44994 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:11.250339 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbedbd2e-3252-4937-8582-df0659b44994 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:11.274992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-bbedbd2e-3252-4937-8582-df0659b44994 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 22:49:11.274992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:11.274992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:11.274992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:11.274992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:11.274992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:11.274992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:11.274992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:11.274992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:11.274992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:11.274992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:11.274992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:11.274992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:11.274992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:11.274992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:11.274992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:11.275973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:11.275973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:11.275973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:11.275973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:11.275973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:11.275973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:11.275973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:11.275973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:11.275973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:11.275973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:11.275973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:11.275973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:11.275973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:11.275973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:11.275973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:11.275973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:11.275973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:11.275973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:11.277331 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:11.277331 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:11.277331 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:11.277331 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:11.277331 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:11.277331 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:11.277331 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:11.277331 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:11.277331 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 22:49:11.277331 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:11.277331 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 249/489] 10.222.0.23 () {56 vars in 1093 bytes} [Mon May 9 22:49:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:11.282307 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d62308-2386-4dc9-8a41-6f8da5f6fb10 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:11.337747 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbedbd2e-3252-4937-8582-df0659b44994 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:11.404246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d62308-2386-4dc9-8a41-6f8da5f6fb10 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:11.437604 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105b69a-38a6-4c1c-831d-9e6eafa52ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:11.438309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105b69a-38a6-4c1c-831d-9e6eafa52ad9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:11.438590 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105b69a-38a6-4c1c-831d-9e6eafa52ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:11.438864 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105b69a-38a6-4c1c-831d-9e6eafa52ad9 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:11.439692 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d105b69a-38a6-4c1c-831d-9e6eafa52ad9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:11.439997 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d105b69a-38a6-4c1c-831d-9e6eafa52ad9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:11.441414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d105b69a-38a6-4c1c-831d-9e6eafa52ad9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:11.463432 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbedbd2e-3252-4937-8582-df0659b44994 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:11.479502 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 241/490] 10.222.0.23 () {56 vars in 1133 bytes} [Mon May 9 22:49:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:11.480775 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163993f5-866d-4954-ae40-687502a278d3 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:11.480987 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163993f5-866d-4954-ae40-687502a278d3 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:11.481266 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163993f5-866d-4954-ae40-687502a278d3 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:11.491376 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 242/491] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:11.491376 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163993f5-866d-4954-ae40-687502a278d3 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:11.507501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf4c16-6a1a-47aa-a6fb-015eb6a5fb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:11.508682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf4c16-6a1a-47aa-a6fb-015eb6a5fb60 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:11.509052 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf4c16-6a1a-47aa-a6fb-015eb6a5fb60 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:11.509414 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf4c16-6a1a-47aa-a6fb-015eb6a5fb60 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:11.522743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf4c16-6a1a-47aa-a6fb-015eb6a5fb60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:11.531868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-afbf4c16-6a1a-47aa-a6fb-015eb6a5fb60 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:11.531868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:11.531868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:11.531868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:11.531868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:11.531868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:11.531868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:11.531868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:11.531868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:11.531868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:11.531868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:11.531868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:11.531868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:11.531868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:11.531868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:11.531868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:11.532860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:11.532860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:11.532860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:11.532860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:11.532860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:11.532860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:11.532860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:11.532860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:11.532860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:11.532860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:11.532860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:11.532860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:11.532860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:11.532860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:11.532860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:11.532860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:11.533665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:11.533665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:11.533665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:11.533665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:11.533665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:11.533665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:11.533665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:11.533665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:11.533665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:11.533665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:11.533665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:11.533665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:11.538228 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 250/492] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 9 22:49:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:11.540112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e901a-e3f3-46fc-b3f5-7e014b14168b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:11.542977 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e901a-e3f3-46fc-b3f5-7e014b14168b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:11.543621 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e901a-e3f3-46fc-b3f5-7e014b14168b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:11.567235 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-860e901a-e3f3-46fc-b3f5-7e014b14168b None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:11.568998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-860e901a-e3f3-46fc-b3f5-7e014b14168b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:11.606543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163993f5-866d-4954-ae40-687502a278d3 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:11.642034 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dbcd71-92e2-4703-9d36-48f4002294e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:11.642947 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbcd71-92e2-4703-9d36-48f4002294e5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:11.643413 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbcd71-92e2-4703-9d36-48f4002294e5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:11.643922 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbcd71-92e2-4703-9d36-48f4002294e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:11.658217 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 251/493] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:49:11.659311 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e901a-e3f3-46fc-b3f5-7e014b14168b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:11.733345 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dbcd71-92e2-4703-9d36-48f4002294e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:11.736007 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dbcd71-92e2-4703-9d36-48f4002294e5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:11.777484 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e901a-e3f3-46fc-b3f5-7e014b14168b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:11.816420 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045405d8-f3ff-448d-8c58-150258846ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:11.817080 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045405d8-f3ff-448d-8c58-150258846ea5 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:11.817362 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045405d8-f3ff-448d-8c58-150258846ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:11.817648 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045405d8-f3ff-448d-8c58-150258846ea5 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:11.818428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045405d8-f3ff-448d-8c58-150258846ea5 None admin] RBAC: Adding query filter params (name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:11.818727 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045405d8-f3ff-448d-8c58-150258846ea5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:11.820087 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045405d8-f3ff-448d-8c58-150258846ea5 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:11.827807 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 252/494] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 9 22:49:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:11.828584 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045405d8-f3ff-448d-8c58-150258846ea5 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:11.843595 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 243/495] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:49:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:49:11.845236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a154273-0b0d-4840-8186-65a125631ba1 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:11.845718 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a154273-0b0d-4840-8186-65a125631ba1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:11.846229 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a154273-0b0d-4840-8186-65a125631ba1 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:11.852053 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 244/496] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:11.852053 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a154273-0b0d-4840-8186-65a125631ba1 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:11.931994 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045405d8-f3ff-448d-8c58-150258846ea5 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:11.957958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a154273-0b0d-4840-8186-65a125631ba1 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:11.966443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2cf14f-be53-4525-b97d-3042028e24f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:11.967732 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cf14f-be53-4525-b97d-3042028e24f8 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:11.968154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cf14f-be53-4525-b97d-3042028e24f8 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:11.968591 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cf14f-be53-4525-b97d-3042028e24f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:11.979963 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2cf14f-be53-4525-b97d-3042028e24f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:11.987841 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-0a2cf14f-be53-4525-b97d-3042028e24f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:11.987841 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:11.987841 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:11.987841 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:11.987841 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:11.987841 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:11.987841 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:11.987841 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:11.987841 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:11.987841 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:11.987841 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:11.987841 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:11.987841 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:11.987841 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:11.987841 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:11.987841 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:11.988731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:11.988731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:11.988731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:11.988731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:11.988731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:11.988731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:11.988731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:11.988731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:11.988731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:11.988731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:11.988731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:11.988731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:11.988731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:11.988731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:11.988731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:11.988731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:11.989513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:11.989513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:11.989513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:11.989513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:11.989513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:11.989513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:11.989513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:11.989513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:11.989513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:11.989513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:11.989513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:11.989513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:11.989513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 253/497] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 9 22:49:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:11.991885 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8a3cd-f7bc-4c1a-a666-3695cf76869a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:11.992669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8a3cd-f7bc-4c1a-a666-3695cf76869a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:11.992949 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8a3cd-f7bc-4c1a-a666-3695cf76869a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:11.993222 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8a3cd-f7bc-4c1a-a666-3695cf76869a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:11.994425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2cf14f-be53-4525-b97d-3042028e24f8 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:12.003284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8a3cd-f7bc-4c1a-a666-3695cf76869a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:12.004766 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8a3cd-f7bc-4c1a-a666-3695cf76869a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:12.022253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-0be8a3cd-f7bc-4c1a-a666-3695cf76869a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:12.022253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:12.022253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:12.022253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:12.022253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:12.022253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:12.022253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:12.022253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:12.022253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:12.022253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:12.022253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:12.022253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:12.022253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:12.022253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:12.022253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:12.023279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:12.023279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:12.023279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:12.023279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:12.023279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:12.023279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:12.023279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:12.023279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:12.023279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:12.023279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:12.023279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:12.023279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:12.023279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:12.023279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:12.023279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:12.023279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:12.023279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:12.024306 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:12.024306 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:12.024306 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:12.024306 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:12.027937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 245/498] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:12.029189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9da73-b2c5-4507-92e1-339210752ba3 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:12.029483 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9da73-b2c5-4507-92e1-339210752ba3 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:12.029767 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9da73-b2c5-4507-92e1-339210752ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:12.046545 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-b6f9da73-b2c5-4507-92e1-339210752ba3 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:12.047759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b6f9da73-b2c5-4507-92e1-339210752ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:12.100180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2cf14f-be53-4525-b97d-3042028e24f8 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:12.126151 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 246/499] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:49:12.126151 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f9da73-b2c5-4507-92e1-339210752ba3 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:12.143704 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce86a6a-19af-47c4-b566-3e39a6501918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:12.144488 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce86a6a-19af-47c4-b566-3e39a6501918 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:12.144764 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce86a6a-19af-47c4-b566-3e39a6501918 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:12.145033 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce86a6a-19af-47c4-b566-3e39a6501918 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:12.155229 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce86a6a-19af-47c4-b566-3e39a6501918 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:12.156687 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce86a6a-19af-47c4-b566-3e39a6501918 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:12.173398 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-dce86a6a-19af-47c4-b566-3e39a6501918 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:12.173398 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:12.173398 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:12.173398 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:12.173398 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:12.173398 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:12.173398 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:12.173398 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:12.173398 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:12.173398 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:12.173398 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:12.173398 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:12.173398 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:12.173398 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:12.173398 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:12.174533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:12.174533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:12.174533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:12.174533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:12.174533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:12.174533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:12.174533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:12.174533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:12.174533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:12.174533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:12.174533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:12.174533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:12.174533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:12.174533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:12.174533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:12.174533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:12.174533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:12.176271 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:12.176271 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:12.176271 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:12.176271 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:12.176271 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 254/500] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:12.176271 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce86a6a-19af-47c4-b566-3e39a6501918 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:12.218402 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f9da73-b2c5-4507-92e1-339210752ba3 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:12.251882 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1740190a-5317-4f84-9dbe-297d8ea85620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:12.252545 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740190a-5317-4f84-9dbe-297d8ea85620 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:12.252852 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740190a-5317-4f84-9dbe-297d8ea85620 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:12.253098 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740190a-5317-4f84-9dbe-297d8ea85620 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:12.253840 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1740190a-5317-4f84-9dbe-297d8ea85620 None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:12.254173 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1740190a-5317-4f84-9dbe-297d8ea85620 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:12.256079 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1740190a-5317-4f84-9dbe-297d8ea85620 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:12.263751 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 247/501] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 9 22:49:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:12.264524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1740190a-5317-4f84-9dbe-297d8ea85620 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:12.277879 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce86a6a-19af-47c4-b566-3e39a6501918 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:12.308937 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc06b3-b552-48bb-aa62-d758d523df45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:12.309512 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc06b3-b552-48bb-aa62-d758d523df45 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:12.309787 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc06b3-b552-48bb-aa62-d758d523df45 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:12.310064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc06b3-b552-48bb-aa62-d758d523df45 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:12.310800 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc06b3-b552-48bb-aa62-d758d523df45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:12.311099 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc06b3-b552-48bb-aa62-d758d523df45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:12.312454 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc06b3-b552-48bb-aa62-d758d523df45 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:12.324227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 255/502] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:12.325000 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc06b3-b552-48bb-aa62-d758d523df45 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:12.357354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1740190a-5317-4f84-9dbe-297d8ea85620 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:12.388616 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d17434-d238-492c-a18d-e9ecde1f02ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:12.389391 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d17434-d238-492c-a18d-e9ecde1f02ef None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:12.389664 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d17434-d238-492c-a18d-e9ecde1f02ef None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:12.389927 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d17434-d238-492c-a18d-e9ecde1f02ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:12.400258 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d17434-d238-492c-a18d-e9ecde1f02ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:12.407810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-76d17434-d238-492c-a18d-e9ecde1f02ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:12.407810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:12.407810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:12.407810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:12.407810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:12.407810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:12.407810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:12.407810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:12.407810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:12.407810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:12.407810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:12.407810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:12.407810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:12.407810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:12.407810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:12.407810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:12.408778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:12.408778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:12.408778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:12.408778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:12.408778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:12.408778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:12.408778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:12.408778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:12.408778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:12.408778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:12.408778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:12.408778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:12.408778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:12.408778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:12.408778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:12.408778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:12.409579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:12.409579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:12.409579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:12.409579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:12.409579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:12.409579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:12.409579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:12.409579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:12.409579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:12.409579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:12.409579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:12.409579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:12.409579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 248/503] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 9 22:49:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:12.410285 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1006634-e921-4a94-8758-53eb8ef576fa None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:12.410464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1006634-e921-4a94-8758-53eb8ef576fa None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:12.411152 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1006634-e921-4a94-8758-53eb8ef576fa None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:12.411956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 249/504] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:12.412643 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1006634-e921-4a94-8758-53eb8ef576fa None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:12.424321 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc06b3-b552-48bb-aa62-d758d523df45 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:12.456408 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b1a59-068d-4bf2-936d-07b5f2914e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:12.457042 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b1a59-068d-4bf2-936d-07b5f2914e74 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:12.457315 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b1a59-068d-4bf2-936d-07b5f2914e74 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:12.457583 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b1a59-068d-4bf2-936d-07b5f2914e74 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:12.458304 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b1a59-068d-4bf2-936d-07b5f2914e74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:12.458598 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b1a59-068d-4bf2-936d-07b5f2914e74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:12.459973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b1a59-068d-4bf2-936d-07b5f2914e74 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:12.470722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 256/505] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:12.471547 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b1a59-068d-4bf2-936d-07b5f2914e74 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:12.509282 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1006634-e921-4a94-8758-53eb8ef576fa None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:12.539941 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503fb474-f94a-4195-9ab7-ec3e27653c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:12.540715 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fb474-f94a-4195-9ab7-ec3e27653c9b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:12.540989 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fb474-f94a-4195-9ab7-ec3e27653c9b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:12.541264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fb474-f94a-4195-9ab7-ec3e27653c9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:12.551219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503fb474-f94a-4195-9ab7-ec3e27653c9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:12.553165 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503fb474-f94a-4195-9ab7-ec3e27653c9b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:12.563092 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b1a59-068d-4bf2-936d-07b5f2914e74 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:12.569874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-503fb474-f94a-4195-9ab7-ec3e27653c9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:12.569874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:12.569874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:12.569874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:12.569874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:12.569874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:12.569874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:12.569874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:12.569874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:12.569874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:12.569874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:12.569874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:12.569874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:12.569874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:12.569874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:12.570914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:12.570914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:12.570914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:12.570914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:12.570914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:12.570914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:12.570914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:12.570914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:12.570914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:12.570914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:12.570914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:12.570914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:12.570914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:12.570914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:12.570914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:12.570914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:12.570914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:12.571979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:12.571979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:12.571979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:12.571979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:12.571979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 250/506] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:12.573209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503fb474-f94a-4195-9ab7-ec3e27653c9b None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:12.595877 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3ecaa-947b-4954-9e27-329855e54ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:12.596654 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3ecaa-947b-4954-9e27-329855e54ff9 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:12.596930 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3ecaa-947b-4954-9e27-329855e54ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:12.597199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3ecaa-947b-4954-9e27-329855e54ff9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:12.604424 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3ecaa-947b-4954-9e27-329855e54ff9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:12.605858 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3ecaa-947b-4954-9e27-329855e54ff9 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:12.620393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-f8f3ecaa-947b-4954-9e27-329855e54ff9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 22:49:12.620393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:12.620393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:12.620393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:12.620393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:12.620393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:12.620393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:12.620393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:12.620393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:12.620393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:12.620393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:12.620393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:12.620393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:12.620393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:12.620393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:12.620393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:12.621464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:12.621464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:12.621464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:12.621464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:12.621464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:12.621464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:12.621464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:12.621464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:12.621464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:12.621464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:12.621464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:12.621464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:12.621464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:12.621464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:12.621464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:12.621464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:12.621464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:12.621464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:12.622445 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:12.622445 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:12.622445 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:12.622445 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:12.622445 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:12.622445 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:12.622445 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:12.622445 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:12.622445 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 22:49:12.622445 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:12.622445 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 257/507] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:49:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:12.623078 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f96664-f123-4a1c-9a10-7e6a3af13ed0 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:12.623360 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f96664-f123-4a1c-9a10-7e6a3af13ed0 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:12.623639 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f96664-f123-4a1c-9a10-7e6a3af13ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:12.640952 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-c8f96664-f123-4a1c-9a10-7e6a3af13ed0 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:12.642141 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c8f96664-f123-4a1c-9a10-7e6a3af13ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:12.671267 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503fb474-f94a-4195-9ab7-ec3e27653c9b None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:12.703182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1335c19-2336-4013-9962-de1ee7164a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:12.703358 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 258/508] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:49:12.703616 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1335c19-2336-4013-9962-de1ee7164a2d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:12.703758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f96664-f123-4a1c-9a10-7e6a3af13ed0 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:12.703890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1335c19-2336-4013-9962-de1ee7164a2d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:12.704155 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1335c19-2336-4013-9962-de1ee7164a2d None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:12.704886 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1335c19-2336-4013-9962-de1ee7164a2d None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:12.705187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1335c19-2336-4013-9962-de1ee7164a2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:12.706601 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1335c19-2336-4013-9962-de1ee7164a2d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:12.719220 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 251/509] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 9 22:49:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:12.720042 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1335c19-2336-4013-9962-de1ee7164a2d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:12.795263 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f96664-f123-4a1c-9a10-7e6a3af13ed0 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:12.814933 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1335c19-2336-4013-9962-de1ee7164a2d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:12.827394 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed3e43-60cb-4351-9413-55b7f04c27e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:12.828168 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed3e43-60cb-4351-9413-55b7f04c27e5 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:12.828438 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed3e43-60cb-4351-9413-55b7f04c27e5 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:12.828703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed3e43-60cb-4351-9413-55b7f04c27e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:12.835013 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed3e43-60cb-4351-9413-55b7f04c27e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:12.836450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed3e43-60cb-4351-9413-55b7f04c27e5 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:12.846233 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a73dcb-063f-4167-81ab-f42e83caaaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:12.846877 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a73dcb-063f-4167-81ab-f42e83caaaff None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:12.847157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a73dcb-063f-4167-81ab-f42e83caaaff None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:12.847436 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a73dcb-063f-4167-81ab-f42e83caaaff None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:12.848144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a73dcb-063f-4167-81ab-f42e83caaaff None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:12.848451 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a73dcb-063f-4167-81ab-f42e83caaaff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:12.850355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a73dcb-063f-4167-81ab-f42e83caaaff None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:12.850623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-17ed3e43-60cb-4351-9413-55b7f04c27e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:12.850623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:12.850623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:12.850623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:12.850623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:12.850623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:12.850623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:12.850623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:12.850623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:12.850623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:12.850623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:12.850623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:12.850623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:12.850623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:12.850623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:12.851558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:12.851558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:12.851558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:12.851558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:12.851558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:12.851558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:12.851558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:12.851558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:12.851558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:12.851558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:12.851558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:12.851558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:12.851558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:12.851558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:12.851558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:12.851558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:12.851558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:12.852598 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:12.852598 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:12.852598 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:12.852598 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:12.852598 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:12.852598 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:12.852598 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:12.852598 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:12.852598 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 259/510] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:12.852598 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed3e43-60cb-4351-9413-55b7f04c27e5 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:12.870478 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 252/511] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:12.871397 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a73dcb-063f-4167-81ab-f42e83caaaff None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:12.946126 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed3e43-60cb-4351-9413-55b7f04c27e5 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:12.961892 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a73dcb-063f-4167-81ab-f42e83caaaff None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:12.976765 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cad37b-0ea2-42d8-ae6a-87b04c16a110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:12.977414 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cad37b-0ea2-42d8-ae6a-87b04c16a110 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:12.977691 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cad37b-0ea2-42d8-ae6a-87b04c16a110 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:12.977959 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cad37b-0ea2-42d8-ae6a-87b04c16a110 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:12.978800 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cad37b-0ea2-42d8-ae6a-87b04c16a110 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:12.979101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cad37b-0ea2-42d8-ae6a-87b04c16a110 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:12.980505 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cad37b-0ea2-42d8-ae6a-87b04c16a110 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:12.994613 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8cdb41-fb1a-4e8a-8b15-a823caeb0bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:12.995419 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cdb41-fb1a-4e8a-8b15-a823caeb0bbf None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:12.995699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cdb41-fb1a-4e8a-8b15-a823caeb0bbf None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:12.995971 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cdb41-fb1a-4e8a-8b15-a823caeb0bbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:13.006174 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8cdb41-fb1a-4e8a-8b15-a823caeb0bbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:13.007599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8cdb41-fb1a-4e8a-8b15-a823caeb0bbf None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:13.016192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 260/512] 10.222.0.23 () {56 vars in 1137 bytes} [Mon May 9 22:49:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:13.016981 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cad37b-0ea2-42d8-ae6a-87b04c16a110 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:13.029242 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-df8cdb41-fb1a-4e8a-8b15-a823caeb0bbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:13.029242 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:13.029242 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:13.029242 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:13.029242 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:13.029242 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:13.029242 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:13.029242 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:13.029242 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:13.029242 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:13.029242 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:13.029242 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:13.029242 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:13.029242 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:13.029242 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:13.030295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:13.030295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:13.030295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:13.030295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:13.030295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:13.030295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:13.030295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:13.030295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:13.030295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:13.030295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:13.030295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:13.030295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:13.030295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:13.030295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:13.030295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:13.030295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:13.030295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:13.031622 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:13.031622 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:13.031622 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:13.031622 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:13.031622 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 253/513] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:13.031622 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8cdb41-fb1a-4e8a-8b15-a823caeb0bbf None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:13.109259 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cad37b-0ea2-42d8-ae6a-87b04c16a110 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:13.125486 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8cdb41-fb1a-4e8a-8b15-a823caeb0bbf None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:13.140454 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ce03c-4cca-4216-a674-7b3909a0d92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:13.141221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ce03c-4cca-4216-a674-7b3909a0d92a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:13.141491 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ce03c-4cca-4216-a674-7b3909a0d92a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:13.141754 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ce03c-4cca-4216-a674-7b3909a0d92a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:13.151643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ce03c-4cca-4216-a674-7b3909a0d92a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:13.153067 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ce03c-4cca-4216-a674-7b3909a0d92a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:13.158032 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63cf0c-a91e-420d-942a-5ce272723a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:13.158700 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63cf0c-a91e-420d-942a-5ce272723a21 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:13.158982 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63cf0c-a91e-420d-942a-5ce272723a21 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:13.159267 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63cf0c-a91e-420d-942a-5ce272723a21 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:13.160032 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63cf0c-a91e-420d-942a-5ce272723a21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:13.160335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63cf0c-a91e-420d-942a-5ce272723a21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:13.161775 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63cf0c-a91e-420d-942a-5ce272723a21 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:13.169630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-ff7ce03c-4cca-4216-a674-7b3909a0d92a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:13.169630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:13.169630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:13.169630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:13.169630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:13.169630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:13.169630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:13.169630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:13.169630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:13.169630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:13.169630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:13.169630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:13.169630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:13.169630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:13.169630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:13.170661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:13.170661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:13.170661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:13.170661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:13.170661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:13.170661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:13.170661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:13.170661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:13.170661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:13.170661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:13.170661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:13.170661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:13.170661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:13.170661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:13.170661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:13.170661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:13.170661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:13.171768 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:13.171768 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:13.171768 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:13.171768 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:13.171768 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 261/514] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:13.171768 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ce03c-4cca-4216-a674-7b3909a0d92a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:13.181452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 254/515] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:13.182296 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63cf0c-a91e-420d-942a-5ce272723a21 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:13.265355 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ce03c-4cca-4216-a674-7b3909a0d92a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:13.274408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63cf0c-a91e-420d-942a-5ce272723a21 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:13.295718 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f77d17-5cfe-49e1-bef1-0a6573968da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:13.296489 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f77d17-5cfe-49e1-bef1-0a6573968da3 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:13.296805 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f77d17-5cfe-49e1-bef1-0a6573968da3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:13.297027 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f77d17-5cfe-49e1-bef1-0a6573968da3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:13.304695 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f77d17-5cfe-49e1-bef1-0a6573968da3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:13.306132 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f77d17-5cfe-49e1-bef1-0a6573968da3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:13.307138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc17749-7904-4611-8c99-f85833d6dbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:13.307771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc17749-7904-4611-8c99-f85833d6dbb0 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:13.308050 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc17749-7904-4611-8c99-f85833d6dbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:13.308321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc17749-7904-4611-8c99-f85833d6dbb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:13.320245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-85f77d17-5cfe-49e1-bef1-0a6573968da3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 09 22:49:13.320245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:13.320245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:13.320245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:13.320245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:13.320245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:13.320245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:13.320245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:13.320245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:13.320245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:13.320245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:13.320245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:13.320245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:13.320245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:13.320245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:13.320245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:13.321236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:13.321236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:13.321236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:13.321236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:13.321236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:13.321236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:13.321236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:13.321236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:13.321236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:13.321236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:13.321236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:13.321236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:13.321236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:13.321236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:13.321236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:13.321236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:13.321236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:13.321236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:13.322187 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:13.322187 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:13.322187 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:13.322187 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:13.322187 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:13.322187 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:13.322187 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:13.322187 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:13.322187 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 09 22:49:13.322187 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:13.322187 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 262/516] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:49:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:13.322187 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f77d17-5cfe-49e1-bef1-0a6573968da3 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:13.375382 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc17749-7904-4611-8c99-f85833d6dbb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:13.376771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc17749-7904-4611-8c99-f85833d6dbb0 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:13.413055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f77d17-5cfe-49e1-bef1-0a6573968da3 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:13.444338 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714bc863-31a7-4487-81e5-2387d01bbacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:13.444980 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714bc863-31a7-4487-81e5-2387d01bbacc None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:13.445255 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714bc863-31a7-4487-81e5-2387d01bbacc None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:13.445523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714bc863-31a7-4487-81e5-2387d01bbacc None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:13.446243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714bc863-31a7-4487-81e5-2387d01bbacc None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:13.446539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714bc863-31a7-4487-81e5-2387d01bbacc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:13.447867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714bc863-31a7-4487-81e5-2387d01bbacc None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:13.459044 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 263/517] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:13.459605 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714bc863-31a7-4487-81e5-2387d01bbacc None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:13.468732 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 255/518] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:49:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:49:13.469610 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc17749-7904-4611-8c99-f85833d6dbb0 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:13.550484 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714bc863-31a7-4487-81e5-2387d01bbacc None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:13.560570 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc17749-7904-4611-8c99-f85833d6dbb0 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:13.582662 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b3bc4-7bb7-4abc-aaf0-2a1989fe8463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:13.583307 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b3bc4-7bb7-4abc-aaf0-2a1989fe8463 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:13.583609 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b3bc4-7bb7-4abc-aaf0-2a1989fe8463 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:13.583885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b3bc4-7bb7-4abc-aaf0-2a1989fe8463 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:13.584593 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b3bc4-7bb7-4abc-aaf0-2a1989fe8463 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:13.584897 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b3bc4-7bb7-4abc-aaf0-2a1989fe8463 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:13.586284 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b3bc4-7bb7-4abc-aaf0-2a1989fe8463 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:13.592351 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8f7b9-ebe6-43c3-b4db-c486ce4dafad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:13.592995 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8f7b9-ebe6-43c3-b4db-c486ce4dafad None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:13.593274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8f7b9-ebe6-43c3-b4db-c486ce4dafad None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:13.593547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8f7b9-ebe6-43c3-b4db-c486ce4dafad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:13.594382 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8f7b9-ebe6-43c3-b4db-c486ce4dafad None admin] RBAC: Adding query filter params (role.id=44847d50e6cb4bc7938c59b4e2d99833, scope.project.id=0500e378ea374ca692e230bbd8781318, user.id=02fcd7cc86134ce488c7dd75dfb174b8) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:13.594694 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8f7b9-ebe6-43c3-b4db-c486ce4dafad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:13.596089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8f7b9-ebe6-43c3-b4db-c486ce4dafad None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:13.597349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 264/519] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:13.598159 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b3bc4-7bb7-4abc-aaf0-2a1989fe8463 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:13.614286 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 256/520] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:49:13 2022] GET /identity/v3/role_assignments?user.id=02fcd7cc86134ce488c7dd75dfb174b8&scope.project.id=0500e378ea374ca692e230bbd8781318&role.id=44847d50e6cb4bc7938c59b4e2d99833 => generated 259 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:13.614286 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8f7b9-ebe6-43c3-b4db-c486ce4dafad None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:13.709024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b3bc4-7bb7-4abc-aaf0-2a1989fe8463 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:13.713492 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8f7b9-ebe6-43c3-b4db-c486ce4dafad None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:13.745579 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2839fd-e490-444e-943f-b9e80732d29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:13.746345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2839fd-e490-444e-943f-b9e80732d29b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:13.746625 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2839fd-e490-444e-943f-b9e80732d29b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:13.746891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2839fd-e490-444e-943f-b9e80732d29b None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:13.747768 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2839fd-e490-444e-943f-b9e80732d29b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:13.748081 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2839fd-e490-444e-943f-b9e80732d29b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:13.749424 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b4c14-b97e-4c63-a0b4-8125a87fc101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:13.749543 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2839fd-e490-444e-943f-b9e80732d29b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:13.750248 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b4c14-b97e-4c63-a0b4-8125a87fc101 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:13.750528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b4c14-b97e-4c63-a0b4-8125a87fc101 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:13.750799 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b4c14-b97e-4c63-a0b4-8125a87fc101 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:13.758450 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b4c14-b97e-4c63-a0b4-8125a87fc101 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:13.759916 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b4c14-b97e-4c63-a0b4-8125a87fc101 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:13.768048 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 265/521] 10.222.0.23 () {56 vars in 1137 bytes} [Mon May 9 22:49:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:13.769247 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2839fd-e490-444e-943f-b9e80732d29b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:13.776279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-a40b4c14-b97e-4c63-a0b4-8125a87fc101 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 09 22:49:13.776279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:13.776279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:13.776279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:13.776279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:13.776279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:13.776279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:13.776279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:13.776279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:13.776279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:13.776279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:13.776279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:13.776279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:13.776279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:13.776279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:13.776279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:13.777260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:13.777260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:13.777260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:13.777260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:13.777260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:13.777260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:13.777260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:13.777260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:13.777260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:13.777260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:13.777260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:13.777260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:13.777260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:13.777260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:13.777260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:13.777260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:13.777260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:13.777260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:13.778201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:13.778201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:13.778201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:13.778201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:13.778201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:13.778201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:13.778201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:13.778201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:13.778201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 09 22:49:13.778201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:13.782103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 257/522] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 9 22:49:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:13.783025 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b4c14-b97e-4c63-a0b4-8125a87fc101 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:13.870495 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2839fd-e490-444e-943f-b9e80732d29b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:13.879366 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b4c14-b97e-4c63-a0b4-8125a87fc101 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:13.903398 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2384d8-4e05-435f-8bf2-dea8dc70f056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:13.904183 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2384d8-4e05-435f-8bf2-dea8dc70f056 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:13.904459 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2384d8-4e05-435f-8bf2-dea8dc70f056 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:13.904728 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2384d8-4e05-435f-8bf2-dea8dc70f056 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:13.912970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849addea-b6b5-4700-b256-c6f83d10448a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:13.913652 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849addea-b6b5-4700-b256-c6f83d10448a None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:13.913932 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849addea-b6b5-4700-b256-c6f83d10448a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:13.914310 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849addea-b6b5-4700-b256-c6f83d10448a None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:13.915016 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849addea-b6b5-4700-b256-c6f83d10448a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:13.915599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2384d8-4e05-435f-8bf2-dea8dc70f056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:13.916467 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849addea-b6b5-4700-b256-c6f83d10448a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:13.917318 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2384d8-4e05-435f-8bf2-dea8dc70f056 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:13.936744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-cd2384d8-4e05-435f-8bf2-dea8dc70f056 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:13.936744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:13.936744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:13.936744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:13.936744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:13.936744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:13.936744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:13.936744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:13.936744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:13.936744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:13.936744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:13.936744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:13.936744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:13.936744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:13.936744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:13.947192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:13.947192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:13.947192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:13.947192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:13.947192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:13.947192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:13.947192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:13.947192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:13.947192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:13.947192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:13.947192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:13.947192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:13.947192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:13.947192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:13.947192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:13.947192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:13.947192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:13.948083 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:13.948083 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:13.948083 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:13.948083 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:13.948294 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 258/523] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 9 22:49:13 2022] POST /identity/v3/users => generated 247 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:49:13.948294 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849addea-b6b5-4700-b256-c6f83d10448a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:13.948428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 266/524] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:13.948428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2384d8-4e05-435f-8bf2-dea8dc70f056 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:14.052183 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2384d8-4e05-435f-8bf2-dea8dc70f056 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:14.078831 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849addea-b6b5-4700-b256-c6f83d10448a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:14.100331 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa229c-05ed-4f61-819a-1c871eda858e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:14.101677 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa229c-05ed-4f61-819a-1c871eda858e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:14.101958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa229c-05ed-4f61-819a-1c871eda858e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:14.102432 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa229c-05ed-4f61-819a-1c871eda858e None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:14.103284 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaa229c-05ed-4f61-819a-1c871eda858e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:14.103594 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaa229c-05ed-4f61-819a-1c871eda858e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:14.105089 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaa229c-05ed-4f61-819a-1c871eda858e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:14.118355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067df4e2-cde0-47a1-abd2-8a1a2e045f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:14.119027 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067df4e2-cde0-47a1-abd2-8a1a2e045f98 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:14.119322 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067df4e2-cde0-47a1-abd2-8a1a2e045f98 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:14.119573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067df4e2-cde0-47a1-abd2-8a1a2e045f98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:14.131649 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 267/525] 10.222.0.23 () {56 vars in 1139 bytes} [Mon May 9 22:49:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:14.131649 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa229c-05ed-4f61-819a-1c871eda858e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:14.192822 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067df4e2-cde0-47a1-abd2-8a1a2e045f98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:14.194321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067df4e2-cde0-47a1-abd2-8a1a2e045f98 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:14.230045 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa229c-05ed-4f61-819a-1c871eda858e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:14.264524 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac85fa5-4329-4893-9cbe-f3ef898933f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:14.265541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac85fa5-4329-4893-9cbe-f3ef898933f7 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:14.266459 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac85fa5-4329-4893-9cbe-f3ef898933f7 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:14.266920 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac85fa5-4329-4893-9cbe-f3ef898933f7 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:14.267828 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac85fa5-4329-4893-9cbe-f3ef898933f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:14.268341 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac85fa5-4329-4893-9cbe-f3ef898933f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:14.269893 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac85fa5-4329-4893-9cbe-f3ef898933f7 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:14.282400 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 268/526] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:14.283553 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac85fa5-4329-4893-9cbe-f3ef898933f7 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:14.296980 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 259/527] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:49:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:49:14.297939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067df4e2-cde0-47a1-abd2-8a1a2e045f98 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:14.383153 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac85fa5-4329-4893-9cbe-f3ef898933f7 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:14.398006 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067df4e2-cde0-47a1-abd2-8a1a2e045f98 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:14.417224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25209f3b-7854-4058-836c-781f905a42ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:14.418237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25209f3b-7854-4058-836c-781f905a42ea None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:14.418735 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25209f3b-7854-4058-836c-781f905a42ea None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:14.419193 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25209f3b-7854-4058-836c-781f905a42ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:14.432190 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b570d-4b09-41b7-8577-2727397420c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:14.432989 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b570d-4b09-41b7-8577-2727397420c5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:14.433265 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b570d-4b09-41b7-8577-2727397420c5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:14.433534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b570d-4b09-41b7-8577-2727397420c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:14.440591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b570d-4b09-41b7-8577-2727397420c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:14.442024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b570d-4b09-41b7-8577-2727397420c5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:14.457158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-6a2b570d-4b09-41b7-8577-2727397420c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:14.457158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:14.457158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:14.457158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:14.457158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:14.457158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:14.457158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:14.457158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:14.457158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:14.457158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:14.457158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:14.457158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:14.457158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:14.457158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:14.457158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:14.458073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:14.458073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:14.458073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:14.458073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:14.458073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:14.458073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:14.458073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:14.458073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:14.458073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:14.458073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:14.458073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:14.458073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:14.458073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:14.458073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:14.458073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:14.458073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:14.458073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:14.459517 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:14.459517 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:14.459517 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:14.459517 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:14.459517 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:14.459517 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:14.459517 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:14.459517 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:14.459517 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 260/528] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:14.460459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b570d-4b09-41b7-8577-2727397420c5 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:14.490869 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25209f3b-7854-4058-836c-781f905a42ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:14.492642 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25209f3b-7854-4058-836c-781f905a42ea None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:14.559499 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b570d-4b09-41b7-8577-2727397420c5 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:14.589677 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 269/529] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:49:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:49:14.590866 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25209f3b-7854-4058-836c-781f905a42ea None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:14.594428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67741ff3-c2b2-4d2a-8191-eab2ab1c4e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:14.594565 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67741ff3-c2b2-4d2a-8191-eab2ab1c4e26 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:14.594846 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67741ff3-c2b2-4d2a-8191-eab2ab1c4e26 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:14.595122 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67741ff3-c2b2-4d2a-8191-eab2ab1c4e26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:14.606017 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67741ff3-c2b2-4d2a-8191-eab2ab1c4e26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:14.607566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67741ff3-c2b2-4d2a-8191-eab2ab1c4e26 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:14.625335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-67741ff3-c2b2-4d2a-8191-eab2ab1c4e26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:14.625335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:14.625335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:14.625335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:14.625335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:14.625335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:14.625335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:14.625335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:14.625335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:14.625335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:14.625335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:14.625335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:14.625335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:14.625335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:14.625335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:14.626291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:14.626291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:14.626291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:14.626291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:14.626291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:14.626291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:14.626291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:14.626291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:14.626291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:14.626291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:14.626291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:14.626291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:14.626291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:14.626291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:14.626291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:14.626291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:14.626291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:14.627132 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:14.627132 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:14.627132 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:14.627132 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:14.627132 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 261/530] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:14.627890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67741ff3-c2b2-4d2a-8191-eab2ab1c4e26 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:14.692209 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25209f3b-7854-4058-836c-781f905a42ea None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:14.728448 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f95df-73f7-4c3b-9be4-8277569614a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:14.729834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f95df-73f7-4c3b-9be4-8277569614a7 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:14.730391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f95df-73f7-4c3b-9be4-8277569614a7 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:14.730862 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f95df-73f7-4c3b-9be4-8277569614a7 None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:14.731058 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67741ff3-c2b2-4d2a-8191-eab2ab1c4e26 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:14.732055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f95df-73f7-4c3b-9be4-8277569614a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:14.732773 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f95df-73f7-4c3b-9be4-8277569614a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:14.735479 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f95df-73f7-4c3b-9be4-8277569614a7 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:14.760104 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 270/531] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:14.761428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f95df-73f7-4c3b-9be4-8277569614a7 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:14.773721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912279f7-6fba-4829-896a-edf08a515305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:14.774964 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912279f7-6fba-4829-896a-edf08a515305 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:14.775247 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912279f7-6fba-4829-896a-edf08a515305 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:14.775520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912279f7-6fba-4829-896a-edf08a515305 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:14.788578 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912279f7-6fba-4829-896a-edf08a515305 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:14.790781 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912279f7-6fba-4829-896a-edf08a515305 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:14.810952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-912279f7-6fba-4829-896a-edf08a515305 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:14.810952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:14.810952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:14.810952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:14.810952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:14.810952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:14.810952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:14.810952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:14.810952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:14.810952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:14.810952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:14.810952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:14.810952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:14.810952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:14.810952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:14.811855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:14.811855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:14.811855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:14.811855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:14.811855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:14.811855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:14.811855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:14.811855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:14.811855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:14.811855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:14.811855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:14.811855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:14.811855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:14.811855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:14.811855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:14.811855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:14.811855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:14.812697 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:14.812697 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:14.812697 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:14.812697 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:14.812697 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 262/532] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:14.813854 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912279f7-6fba-4829-896a-edf08a515305 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:14.883222 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f95df-73f7-4c3b-9be4-8277569614a7 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:14.916168 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fe541-dbc1-40a2-8ec6-9439fcd2c8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:14.916829 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fe541-dbc1-40a2-8ec6-9439fcd2c8f1 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:14.917108 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fe541-dbc1-40a2-8ec6-9439fcd2c8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:14.917378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fe541-dbc1-40a2-8ec6-9439fcd2c8f1 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:14.918176 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fe541-dbc1-40a2-8ec6-9439fcd2c8f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:14.918487 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fe541-dbc1-40a2-8ec6-9439fcd2c8f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:14.919895 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fe541-dbc1-40a2-8ec6-9439fcd2c8f1 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:14.932132 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 271/533] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:14.932359 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912279f7-6fba-4829-896a-edf08a515305 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:14.932940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fe541-dbc1-40a2-8ec6-9439fcd2c8f1 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:14.974676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b46c6-28ef-4d4d-b1b6-7a8d0a2c4479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:14.975532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b46c6-28ef-4d4d-b1b6-7a8d0a2c4479 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:14.975814 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b46c6-28ef-4d4d-b1b6-7a8d0a2c4479 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:14.976091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b46c6-28ef-4d4d-b1b6-7a8d0a2c4479 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:14.977169 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b46c6-28ef-4d4d-b1b6-7a8d0a2c4479 None admin] RBAC: Adding query filter params (role.id=44847d50e6cb4bc7938c59b4e2d99833, scope.project.id=0500e378ea374ca692e230bbd8781318, user.id=5fe668a2cee040c7a56d1f8685744cd1) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:14.977864 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b46c6-28ef-4d4d-b1b6-7a8d0a2c4479 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:14.979742 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b46c6-28ef-4d4d-b1b6-7a8d0a2c4479 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:14.987132 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 263/534] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:49:14 2022] GET /identity/v3/role_assignments?user.id=5fe668a2cee040c7a56d1f8685744cd1&scope.project.id=0500e378ea374ca692e230bbd8781318&role.id=44847d50e6cb4bc7938c59b4e2d99833 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:14.988047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b46c6-28ef-4d4d-b1b6-7a8d0a2c4479 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:15.105219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b46c6-28ef-4d4d-b1b6-7a8d0a2c4479 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:15.112248 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fe541-dbc1-40a2-8ec6-9439fcd2c8f1 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:15.173100 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc5c3c-eb12-4203-83f7-9f6d3beeafdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:15.173919 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc5c3c-eb12-4203-83f7-9f6d3beeafdf None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:15.174219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc5c3c-eb12-4203-83f7-9f6d3beeafdf None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:15.174493 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc5c3c-eb12-4203-83f7-9f6d3beeafdf None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:15.187119 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6363ef6-a658-46f1-a712-aa1c00b425e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:15.187811 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6363ef6-a658-46f1-a712-aa1c00b425e6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:15.188088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6363ef6-a658-46f1-a712-aa1c00b425e6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:15.188372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6363ef6-a658-46f1-a712-aa1c00b425e6 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:15.189108 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6363ef6-a658-46f1-a712-aa1c00b425e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:15.189413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6363ef6-a658-46f1-a712-aa1c00b425e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:15.190524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc5c3c-eb12-4203-83f7-9f6d3beeafdf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:15.192024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc5c3c-eb12-4203-83f7-9f6d3beeafdf None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:15.194500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6363ef6-a658-46f1-a712-aa1c00b425e6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:15.220576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-1ebc5c3c-eb12-4203-83f7-9f6d3beeafdf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:15.220576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:15.220576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:15.220576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:15.220576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:15.220576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:15.220576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:15.220576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:15.220576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:15.220576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:15.220576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:15.220576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:15.220576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:15.220576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:15.220576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:15.222682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:15.222682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:15.222682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:15.222682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:15.222682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:15.222682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:15.222682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:15.222682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:15.222682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:15.222682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:15.222682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:15.222682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:15.222682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:15.222682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:15.222682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:15.222682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:15.222682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:15.225001 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:15.225001 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:15.225001 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:15.225001 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:15.225001 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:15.225001 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:15.225001 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:15.225001 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:15.225962 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 272/535] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:15.226101 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 264/536] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:15.229174 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6363ef6-a658-46f1-a712-aa1c00b425e6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:15.235503 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc5c3c-eb12-4203-83f7-9f6d3beeafdf None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:15.334510 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6363ef6-a658-46f1-a712-aa1c00b425e6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:15.352052 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc5c3c-eb12-4203-83f7-9f6d3beeafdf None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:15.368936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d5dff-dbaf-4ad7-bfa4-4dd1c4cbc8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:15.369729 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d5dff-dbaf-4ad7-bfa4-4dd1c4cbc8f5 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:15.370012 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d5dff-dbaf-4ad7-bfa4-4dd1c4cbc8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:15.370317 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d5dff-dbaf-4ad7-bfa4-4dd1c4cbc8f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:15.377244 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d5dff-dbaf-4ad7-bfa4-4dd1c4cbc8f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:15.378835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d5dff-dbaf-4ad7-bfa4-4dd1c4cbc8f5 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:15.393225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-335d5dff-dbaf-4ad7-bfa4-4dd1c4cbc8f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:15.393225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:15.393225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:15.393225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:15.393225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:15.393225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:15.393225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:15.393225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:15.393225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:15.393225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:15.393225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:15.393225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:15.393225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:15.393225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:15.393225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:15.395286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:15.395286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:15.395286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:15.395286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:15.395286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:15.395286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:15.395286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:15.395286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:15.395286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:15.395286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:15.395286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:15.395286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:15.395286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:15.395286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:15.395286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:15.395286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:15.395286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:15.397877 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:15.397877 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:15.397877 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:15.397877 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:15.397877 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:15.397877 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:15.397877 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:15.397877 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:15.397877 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 273/537] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:15.398973 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be933e1f-9b4e-47c5-80f4-cfd1239e46ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:15.398973 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be933e1f-9b4e-47c5-80f4-cfd1239e46ed None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:15.398973 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be933e1f-9b4e-47c5-80f4-cfd1239e46ed None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:15.398973 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be933e1f-9b4e-47c5-80f4-cfd1239e46ed None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:15.399517 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be933e1f-9b4e-47c5-80f4-cfd1239e46ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:15.399517 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be933e1f-9b4e-47c5-80f4-cfd1239e46ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:15.400021 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be933e1f-9b4e-47c5-80f4-cfd1239e46ed None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:15.406140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d5dff-dbaf-4ad7-bfa4-4dd1c4cbc8f5 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:15.412525 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 265/538] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:15.429721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be933e1f-9b4e-47c5-80f4-cfd1239e46ed None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:15.507610 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d5dff-dbaf-4ad7-bfa4-4dd1c4cbc8f5 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:15.541235 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c153a-aef9-4c14-bdcd-9d02f4e0b9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:15.541901 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c153a-aef9-4c14-bdcd-9d02f4e0b9fa None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:15.542207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c153a-aef9-4c14-bdcd-9d02f4e0b9fa None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:15.542485 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c153a-aef9-4c14-bdcd-9d02f4e0b9fa None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:15.543282 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c153a-aef9-4c14-bdcd-9d02f4e0b9fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:15.543573 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c153a-aef9-4c14-bdcd-9d02f4e0b9fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:15.545015 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c153a-aef9-4c14-bdcd-9d02f4e0b9fa None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:15.562227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 274/539] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:15.563823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c153a-aef9-4c14-bdcd-9d02f4e0b9fa None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:15.610670 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be933e1f-9b4e-47c5-80f4-cfd1239e46ed None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:15.665765 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c153a-aef9-4c14-bdcd-9d02f4e0b9fa None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:15.682825 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b441281-8a45-4df3-8285-d8f0bb46c53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:15.683490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b441281-8a45-4df3-8285-d8f0bb46c53a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:15.683771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b441281-8a45-4df3-8285-d8f0bb46c53a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:15.684046 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b441281-8a45-4df3-8285-d8f0bb46c53a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:15.684868 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b441281-8a45-4df3-8285-d8f0bb46c53a None admin] RBAC: Adding query filter params (role.id=44847d50e6cb4bc7938c59b4e2d99833, scope.project.id=0500e378ea374ca692e230bbd8781318, user.id=a81caf0558f2449cbbc8f4cd9446f363) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:15.685179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b441281-8a45-4df3-8285-d8f0bb46c53a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:15.686647 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b441281-8a45-4df3-8285-d8f0bb46c53a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:15.699127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f7c19-3c97-4559-891c-b6553115c783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:15.699833 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f7c19-3c97-4559-891c-b6553115c783 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:15.700115 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f7c19-3c97-4559-891c-b6553115c783 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:15.700395 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f7c19-3c97-4559-891c-b6553115c783 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:15.701219 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f7c19-3c97-4559-891c-b6553115c783 None admin] RBAC: Adding query filter params (role.id=44847d50e6cb4bc7938c59b4e2d99833, scope.project.id=0500e378ea374ca692e230bbd8781318, user.id=96fd3f6dd9e74ba7a4ad159e43e8ae96) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:15.701528 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f7c19-3c97-4559-891c-b6553115c783 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:15.702980 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f7c19-3c97-4559-891c-b6553115c783 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:15.703530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 266/540] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:49:15 2022] GET /identity/v3/role_assignments?user.id=a81caf0558f2449cbbc8f4cd9446f363&scope.project.id=0500e378ea374ca692e230bbd8781318&role.id=44847d50e6cb4bc7938c59b4e2d99833 => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:15.715237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 275/541] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:49:15 2022] GET /identity/v3/role_assignments?user.id=96fd3f6dd9e74ba7a4ad159e43e8ae96&scope.project.id=0500e378ea374ca692e230bbd8781318&role.id=44847d50e6cb4bc7938c59b4e2d99833 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:15.931339 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6baf01-b2bf-46f3-9e86-29df966f31ab None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:15.931723 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6baf01-b2bf-46f3-9e86-29df966f31ab None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:15.931897 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6baf01-b2bf-46f3-9e86-29df966f31ab None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:15.933248 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 267/542] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:15.940299 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbf5d7-3320-4988-bdf1-c9cd77bd065e None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:15.940627 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbf5d7-3320-4988-bdf1-c9cd77bd065e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:15.940851 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbf5d7-3320-4988-bdf1-c9cd77bd065e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:15.963395 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-55bbf5d7-3320-4988-bdf1-c9cd77bd065e None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:15.964667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-55bbf5d7-3320-4988-bdf1-c9cd77bd065e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:16.051262 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 276/543] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:49:16.074342 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6baf01-b2bf-46f3-9e86-29df966f31ab None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:16.200594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6baf01-b2bf-46f3-9e86-29df966f31ab None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:16.233691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d41084-35a5-4860-a313-36730defb414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:16.234808 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d41084-35a5-4860-a313-36730defb414 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:16.235083 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d41084-35a5-4860-a313-36730defb414 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:16.235348 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d41084-35a5-4860-a313-36730defb414 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:16.251033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d41084-35a5-4860-a313-36730defb414 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:16.258752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-d5d41084-35a5-4860-a313-36730defb414 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:16.258752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:16.258752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:16.258752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:16.258752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:16.258752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:16.258752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:16.258752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:16.258752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:16.258752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:16.258752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:16.258752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:16.258752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:16.258752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:16.258752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:16.258752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:16.260573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:16.260573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:16.260573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:16.260573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:16.260573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:16.260573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:16.260573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:16.260573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:16.260573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:16.260573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:16.260573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:16.260573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:16.260573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:16.260573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:16.260573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:16.260573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:16.262284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:16.262284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:16.262284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:16.262284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:16.262284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:16.262284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:16.262284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:16.262284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:16.262284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:16.262284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:16.262284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:16.262284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:16.262284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 268/544] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 9 22:49:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:16.273075 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bbf5d7-3320-4988-bdf1-c9cd77bd065e None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:16.412154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bbf5d7-3320-4988-bdf1-c9cd77bd065e None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:16.454180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16c504-ddb4-4b2d-898d-a117c335fa63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:16.454776 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16c504-ddb4-4b2d-898d-a117c335fa63 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:16.455054 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16c504-ddb4-4b2d-898d-a117c335fa63 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:16.455326 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16c504-ddb4-4b2d-898d-a117c335fa63 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:16.456070 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea16c504-ddb4-4b2d-898d-a117c335fa63 None admin] RBAC: Adding query filter params (name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:16.456376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea16c504-ddb4-4b2d-898d-a117c335fa63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:16.457747 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea16c504-ddb4-4b2d-898d-a117c335fa63 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:16.466880 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 277/545] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 9 22:49:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:16.472551 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d41084-35a5-4860-a313-36730defb414 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:16.600227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd57f16-61dd-4bd1-8919-68fcb1509b9e None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:16.600548 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd57f16-61dd-4bd1-8919-68fcb1509b9e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:16.600779 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd57f16-61dd-4bd1-8919-68fcb1509b9e None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:16.602404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 278/546] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:16.624251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597de80-5c12-4628-b46d-a28f62f9cdf9 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:16.624251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597de80-5c12-4628-b46d-a28f62f9cdf9 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:16.624618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597de80-5c12-4628-b46d-a28f62f9cdf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:16.638307 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d41084-35a5-4860-a313-36730defb414 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:16.665481 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-6597de80-5c12-4628-b46d-a28f62f9cdf9 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:16.666694 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-6597de80-5c12-4628-b46d-a28f62f9cdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:16.676103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356ab76-f997-49aa-946c-c6afc029456d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:16.677234 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356ab76-f997-49aa-946c-c6afc029456d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:16.678286 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356ab76-f997-49aa-946c-c6afc029456d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:16.678821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356ab76-f997-49aa-946c-c6afc029456d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:16.689946 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356ab76-f997-49aa-946c-c6afc029456d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:16.691713 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356ab76-f997-49aa-946c-c6afc029456d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:16.710556 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-c356ab76-f997-49aa-946c-c6afc029456d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:16.710556 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:16.710556 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:16.710556 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:16.710556 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:16.710556 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:16.710556 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:16.710556 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:16.710556 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:16.710556 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:16.710556 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:16.710556 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:16.710556 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:16.710556 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:16.710556 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:16.711428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:16.711428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:16.711428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:16.711428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:16.711428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:16.711428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:16.711428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:16.711428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:16.711428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:16.711428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:16.711428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:16.711428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:16.711428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:16.711428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:16.711428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:16.711428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:16.711428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:16.712286 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:16.712286 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:16.712286 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:16.712286 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:16.714404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 269/547] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:16.720127 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356ab76-f997-49aa-946c-c6afc029456d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:16.811544 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 279/548] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:49:16.820992 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356ab76-f997-49aa-946c-c6afc029456d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:16.835992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597de80-5c12-4628-b46d-a28f62f9cdf9 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:16.868974 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141c532-e1e3-4072-8015-d90eb39d0f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:16.869989 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141c532-e1e3-4072-8015-d90eb39d0f9b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:16.870493 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141c532-e1e3-4072-8015-d90eb39d0f9b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:16.871941 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141c532-e1e3-4072-8015-d90eb39d0f9b None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:16.872887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141c532-e1e3-4072-8015-d90eb39d0f9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:16.873590 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141c532-e1e3-4072-8015-d90eb39d0f9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:16.876986 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141c532-e1e3-4072-8015-d90eb39d0f9b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:16.897042 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 270/549] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:16.907362 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141c532-e1e3-4072-8015-d90eb39d0f9b None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:17.009438 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597de80-5c12-4628-b46d-a28f62f9cdf9 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:17.076151 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8f359-fd3c-4ff5-b49e-4ce36641135a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:17.077304 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8f359-fd3c-4ff5-b49e-4ce36641135a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:17.077837 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8f359-fd3c-4ff5-b49e-4ce36641135a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:17.078329 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8f359-fd3c-4ff5-b49e-4ce36641135a None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:17.093304 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141c532-e1e3-4072-8015-d90eb39d0f9b None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:17.097606 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8f359-fd3c-4ff5-b49e-4ce36641135a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:17.107780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-aeb8f359-fd3c-4ff5-b49e-4ce36641135a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:17.107780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:17.107780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:17.107780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:17.107780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:17.107780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:17.107780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:17.107780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:17.107780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:17.107780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:17.107780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:17.107780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:17.107780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:17.107780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:17.107780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:17.107780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:17.108655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:17.108655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:17.108655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:17.108655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:17.108655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:17.108655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:17.108655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:17.108655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:17.108655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:17.108655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:17.108655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:17.108655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:17.108655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:17.108655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:17.108655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:17.108655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:17.109700 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:17.109700 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:17.109700 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:17.109700 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:17.109700 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:17.109700 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:17.109700 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:17.109700 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:17.109700 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:17.109700 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:17.109700 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:17.109700 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:17.113142 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 280/550] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 9 22:49:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:17.119924 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8f359-fd3c-4ff5-b49e-4ce36641135a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:17.154106 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1625b-e2e3-4e3a-bbd1-bbf064013c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:17.154914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1625b-e2e3-4e3a-bbd1-bbf064013c73 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:17.155193 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1625b-e2e3-4e3a-bbd1-bbf064013c73 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:17.155463 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1625b-e2e3-4e3a-bbd1-bbf064013c73 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:17.168744 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c1625b-e2e3-4e3a-bbd1-bbf064013c73 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:17.170255 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c1625b-e2e3-4e3a-bbd1-bbf064013c73 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:17.195120 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-74c1625b-e2e3-4e3a-bbd1-bbf064013c73 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 22:49:17.195120 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:17.195120 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:17.195120 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:17.195120 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:17.195120 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:17.195120 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:17.195120 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:17.195120 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:17.195120 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:17.195120 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:17.195120 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:17.195120 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:17.195120 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:17.195120 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:17.195120 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:17.196081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:17.196081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:17.196081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:17.196081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:17.196081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:17.196081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:17.196081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:17.196081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:17.196081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:17.196081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:17.196081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:17.196081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:17.196081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:17.196081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:17.196081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:17.196081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:17.196081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:17.196081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:17.197381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:17.197381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:17.197381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:17.197381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:17.197381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:17.197381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:17.197381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:17.197381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:17.197381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 22:49:17.197381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:17.197381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 271/551] 10.222.0.23 () {56 vars in 1093 bytes} [Mon May 9 22:49:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:17.211109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1625b-e2e3-4e3a-bbd1-bbf064013c73 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:17.320906 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1625b-e2e3-4e3a-bbd1-bbf064013c73 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:17.329754 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8f359-fd3c-4ff5-b49e-4ce36641135a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:17.360952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc107a09-842e-4fdd-ba7e-4ab9fa05e38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:17.361983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc107a09-842e-4fdd-ba7e-4ab9fa05e38e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:17.362512 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc107a09-842e-4fdd-ba7e-4ab9fa05e38e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:17.363017 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc107a09-842e-4fdd-ba7e-4ab9fa05e38e None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:17.364056 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc107a09-842e-4fdd-ba7e-4ab9fa05e38e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:17.364567 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc107a09-842e-4fdd-ba7e-4ab9fa05e38e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:17.366399 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc107a09-842e-4fdd-ba7e-4ab9fa05e38e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:17.383468 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4e3cb-ae63-4029-b4ac-98e681e15138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:17.384148 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4e3cb-ae63-4029-b4ac-98e681e15138 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:17.384428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4e3cb-ae63-4029-b4ac-98e681e15138 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:17.384697 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4e3cb-ae63-4029-b4ac-98e681e15138 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:17.385442 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4e3cb-ae63-4029-b4ac-98e681e15138 None admin] RBAC: Adding query filter params (name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:17.385754 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4e3cb-ae63-4029-b4ac-98e681e15138 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:17.388989 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4e3cb-ae63-4029-b4ac-98e681e15138 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:17.402327 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 272/552] 10.222.0.23 () {56 vars in 1133 bytes} [Mon May 9 22:49:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:17.404504 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 281/553] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 9 22:49:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:17.411202 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc107a09-842e-4fdd-ba7e-4ab9fa05e38e None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:17.415112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4e3cb-ae63-4029-b4ac-98e681e15138 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:17.531815 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc107a09-842e-4fdd-ba7e-4ab9fa05e38e None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:17.544950 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4e3cb-ae63-4029-b4ac-98e681e15138 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:17.566222 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31055c7-4953-43db-a605-917ff9534df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:17.567036 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31055c7-4953-43db-a605-917ff9534df8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:17.567329 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31055c7-4953-43db-a605-917ff9534df8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:17.567555 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31055c7-4953-43db-a605-917ff9534df8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:17.578226 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31055c7-4953-43db-a605-917ff9534df8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:17.579484 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31055c7-4953-43db-a605-917ff9534df8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:17.593530 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a0a63-28fe-487b-a986-77687c2ef126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:17.594638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a0a63-28fe-487b-a986-77687c2ef126 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:17.595114 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a0a63-28fe-487b-a986-77687c2ef126 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:17.595606 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a0a63-28fe-487b-a986-77687c2ef126 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:17.597759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-e31055c7-4953-43db-a605-917ff9534df8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:17.597759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:17.597759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:17.597759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:17.597759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:17.597759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:17.597759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:17.597759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:17.597759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:17.597759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:17.597759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:17.597759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:17.597759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:17.597759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:17.597759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:17.598620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:17.598620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:17.598620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:17.598620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:17.598620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:17.598620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:17.598620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:17.598620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:17.598620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:17.598620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:17.598620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:17.598620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:17.598620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:17.598620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:17.598620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:17.598620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:17.598620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:17.599497 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:17.599497 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:17.599497 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:17.599497 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:17.599497 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 273/554] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:17.609190 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31055c7-4953-43db-a605-917ff9534df8 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:17.633036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a0a63-28fe-487b-a986-77687c2ef126 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:17.640052 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a0a63-28fe-487b-a986-77687c2ef126 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:17.678517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-869a0a63-28fe-487b-a986-77687c2ef126 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:17.678517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:17.678517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:17.678517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:17.678517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:17.678517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:17.678517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:17.678517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:17.678517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:17.678517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:17.678517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:17.678517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:17.678517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:17.678517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:17.678517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:17.679430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:17.679430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:17.679430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:17.679430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:17.679430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:17.679430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:17.679430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:17.679430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:17.679430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:17.679430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:17.679430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:17.679430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:17.679430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:17.679430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:17.679430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:17.679430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:17.679430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:17.681698 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:17.681698 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:17.681698 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:17.681698 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:17.681698 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 282/555] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:17.687744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a0a63-28fe-487b-a986-77687c2ef126 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:17.717047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31055c7-4953-43db-a605-917ff9534df8 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:17.757238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa39bd5-0226-4630-a7c8-12b04803e111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:17.757965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa39bd5-0226-4630-a7c8-12b04803e111 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:17.758284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa39bd5-0226-4630-a7c8-12b04803e111 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:17.758559 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa39bd5-0226-4630-a7c8-12b04803e111 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:17.759281 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa39bd5-0226-4630-a7c8-12b04803e111 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:17.759586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa39bd5-0226-4630-a7c8-12b04803e111 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:17.761007 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa39bd5-0226-4630-a7c8-12b04803e111 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:17.774265 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 274/556] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:17.779626 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa39bd5-0226-4630-a7c8-12b04803e111 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:17.888817 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a0a63-28fe-487b-a986-77687c2ef126 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:17.889994 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa39bd5-0226-4630-a7c8-12b04803e111 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:17.922324 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdb22b-0bf1-456b-86da-e80aee4e5bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:17.922992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdb22b-0bf1-456b-86da-e80aee4e5bd2 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:17.923271 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdb22b-0bf1-456b-86da-e80aee4e5bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:17.923547 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdb22b-0bf1-456b-86da-e80aee4e5bd2 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:17.924267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cdb22b-0bf1-456b-86da-e80aee4e5bd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:17.924573 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cdb22b-0bf1-456b-86da-e80aee4e5bd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:17.925935 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cdb22b-0bf1-456b-86da-e80aee4e5bd2 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:17.931709 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f69e6e9-1a42-46e9-9b74-ccccd5197c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:17.932503 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69e6e9-1a42-46e9-9b74-ccccd5197c01 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:17.933300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69e6e9-1a42-46e9-9b74-ccccd5197c01 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:17.933593 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69e6e9-1a42-46e9-9b74-ccccd5197c01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:17.938177 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 283/557] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:17.939554 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203becd7-598d-42bc-9b63-b3b3885942e8 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:17.939842 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203becd7-598d-42bc-9b63-b3b3885942e8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:17.940116 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203becd7-598d-42bc-9b63-b3b3885942e8 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:17.941377 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f69e6e9-1a42-46e9-9b74-ccccd5197c01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:17.941646 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 284/558] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:17.942878 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f69e6e9-1a42-46e9-9b74-ccccd5197c01 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:17.946409 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203becd7-598d-42bc-9b63-b3b3885942e8 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:17.960689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-2f69e6e9-1a42-46e9-9b74-ccccd5197c01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:17.960689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:17.960689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:17.960689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:17.960689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:17.960689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:17.960689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:17.960689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:17.960689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:17.960689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:17.960689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:17.960689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:17.960689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:17.960689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:17.960689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:17.961672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:17.961672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:17.961672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:17.961672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:17.961672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:17.961672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:17.961672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:17.961672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:17.961672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:17.961672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:17.961672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:17.961672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:17.961672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:17.961672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:17.961672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:17.961672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:17.961672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:17.962547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:17.962547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:17.962547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:17.962547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:17.962547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:17.962547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:17.962547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:17.962547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:17.962547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 275/559] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:17.966249 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5afb0-c380-4c2c-8ee6-c503429dc66b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:17.966249 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5afb0-c380-4c2c-8ee6-c503429dc66b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:17.966249 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5afb0-c380-4c2c-8ee6-c503429dc66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:17.986906 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-b0b5afb0-c380-4c2c-8ee6-c503429dc66b None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:17.989037 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b0b5afb0-c380-4c2c-8ee6-c503429dc66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:18.054350 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203becd7-598d-42bc-9b63-b3b3885942e8 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:18.061699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 276/560] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:49:18.062570 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5afb0-c380-4c2c-8ee6-c503429dc66b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:18.114611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16284c40-6ef6-4842-ba54-b1e5fa8f9373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:18.115434 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16284c40-6ef6-4842-ba54-b1e5fa8f9373 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:18.115713 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16284c40-6ef6-4842-ba54-b1e5fa8f9373 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:18.115986 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16284c40-6ef6-4842-ba54-b1e5fa8f9373 None admin] PATH_INFO: `/v3/users/swift` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:18.123670 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16284c40-6ef6-4842-ba54-b1e5fa8f9373 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:18.125116 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16284c40-6ef6-4842-ba54-b1e5fa8f9373 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:18.140601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-16284c40-6ef6-4842-ba54-b1e5fa8f9373 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 22:49:18.140601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:18.140601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:18.140601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:18.140601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:18.140601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:18.140601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:18.140601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:18.140601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:18.140601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:18.140601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:18.140601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:18.140601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:18.140601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:18.140601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:18.140601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:18.142748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:18.142748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:18.142748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:18.142748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:18.142748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:18.142748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:18.142748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:18.142748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:18.142748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:18.142748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:18.142748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:18.142748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:18.142748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:18.142748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:18.142748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:18.142748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:18.142748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:18.142748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:18.145301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:18.145301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:18.145301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:18.145301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:18.145301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:18.145301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:18.145301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:18.145301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:18.145301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 22:49:18.145301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:18.145301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 285/561] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:49:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:18.145301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16284c40-6ef6-4842-ba54-b1e5fa8f9373 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:18.164793 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5afb0-c380-4c2c-8ee6-c503429dc66b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:18.199140 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5cf38-78f2-475e-bdec-76c748ba7a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:18.199825 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5cf38-78f2-475e-bdec-76c748ba7a79 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:18.200104 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5cf38-78f2-475e-bdec-76c748ba7a79 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:18.200376 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5cf38-78f2-475e-bdec-76c748ba7a79 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:18.201183 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5cf38-78f2-475e-bdec-76c748ba7a79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:18.201490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5cf38-78f2-475e-bdec-76c748ba7a79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:18.203094 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5cf38-78f2-475e-bdec-76c748ba7a79 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:18.216710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 277/562] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:18.217691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5cf38-78f2-475e-bdec-76c748ba7a79 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:18.260265 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16284c40-6ef6-4842-ba54-b1e5fa8f9373 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:18.296616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedec397-8620-44e3-83e6-0d20561bf106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:18.297660 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedec397-8620-44e3-83e6-0d20561bf106 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:18.298091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedec397-8620-44e3-83e6-0d20561bf106 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:18.298536 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedec397-8620-44e3-83e6-0d20561bf106 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:18.309514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedec397-8620-44e3-83e6-0d20561bf106 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:18.317655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-dedec397-8620-44e3-83e6-0d20561bf106 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:18.317655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:18.317655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:18.317655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:18.317655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:18.317655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:18.317655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:18.317655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:18.317655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:18.317655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:18.317655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:18.317655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:18.317655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:18.317655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:18.317655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:18.317655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:18.319300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:18.319300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:18.319300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:18.319300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:18.319300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:18.319300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:18.319300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:18.319300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:18.319300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:18.319300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:18.319300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:18.319300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:18.319300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:18.319300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:18.319300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:18.319300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:18.320123 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:18.320123 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:18.320123 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:18.320123 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:18.320123 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:18.320123 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:18.320123 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:18.320123 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:18.320123 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:18.320123 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:18.320123 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:18.320123 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:18.321978 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5cf38-78f2-475e-bdec-76c748ba7a79 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:18.327110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 286/563] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 9 22:49:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:18.328142 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedec397-8620-44e3-83e6-0d20561bf106 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:18.357647 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f9c30-11ae-4960-8780-ab77998006a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:18.358357 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f9c30-11ae-4960-8780-ab77998006a0 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:18.358629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f9c30-11ae-4960-8780-ab77998006a0 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:18.358897 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f9c30-11ae-4960-8780-ab77998006a0 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:18.360009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f9c30-11ae-4960-8780-ab77998006a0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:18.360317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f9c30-11ae-4960-8780-ab77998006a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:18.361886 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f9c30-11ae-4960-8780-ab77998006a0 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:18.384643 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 278/564] 10.222.0.23 () {56 vars in 1135 bytes} [Mon May 9 22:49:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:18.384643 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f9c30-11ae-4960-8780-ab77998006a0 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:18.445425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedec397-8620-44e3-83e6-0d20561bf106 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:18.478670 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245c8ec-9fb7-45da-8969-6a9754fbb48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:18.479550 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245c8ec-9fb7-45da-8969-6a9754fbb48a None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:18.479822 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245c8ec-9fb7-45da-8969-6a9754fbb48a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:18.480089 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245c8ec-9fb7-45da-8969-6a9754fbb48a None admin] PATH_INFO: `/v3/projects/0500e378ea374ca692e230bbd8781318/users/02fcd7cc86134ce488c7dd75dfb174b8/roles/44847d50e6cb4bc7938c59b4e2d99833` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:18.496232 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245c8ec-9fb7-45da-8969-6a9754fbb48a None admin] RBAC: Authorizing `identity:create_grant(project_id=0500e378ea374ca692e230bbd8781318, user_id=02fcd7cc86134ce488c7dd75dfb174b8, role_id=44847d50e6cb4bc7938c59b4e2d99833)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:18.497517 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f9c30-11ae-4960-8780-ab77998006a0 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:18.497690 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245c8ec-9fb7-45da-8969-6a9754fbb48a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:18.511787 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 287/565] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:49:18 2022] PUT /identity/v3/projects/0500e378ea374ca692e230bbd8781318/users/02fcd7cc86134ce488c7dd75dfb174b8/roles/44847d50e6cb4bc7938c59b4e2d99833 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:49:18.523113 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245c8ec-9fb7-45da-8969-6a9754fbb48a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:18.542178 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71697d6f-7864-4ff0-b5b7-081f1e525433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:18.542890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71697d6f-7864-4ff0-b5b7-081f1e525433 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:18.543172 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71697d6f-7864-4ff0-b5b7-081f1e525433 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:18.543506 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71697d6f-7864-4ff0-b5b7-081f1e525433 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:18.544293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71697d6f-7864-4ff0-b5b7-081f1e525433 None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:18.544647 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71697d6f-7864-4ff0-b5b7-081f1e525433 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:18.546951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71697d6f-7864-4ff0-b5b7-081f1e525433 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:18.562502 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 279/566] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 9 22:49:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:18.564452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffecee4-10f2-4831-b72e-908f2da6868a None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:18.565804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffecee4-10f2-4831-b72e-908f2da6868a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:18.567125 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffecee4-10f2-4831-b72e-908f2da6868a None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:18.569280 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 280/567] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:18.569886 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffecee4-10f2-4831-b72e-908f2da6868a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:18.657854 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245c8ec-9fb7-45da-8969-6a9754fbb48a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:18.684834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffecee4-10f2-4831-b72e-908f2da6868a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:18.697942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23eaad-02a7-453a-bac9-e08c2b0c88dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:18.698636 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23eaad-02a7-453a-bac9-e08c2b0c88dc None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:18.698920 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23eaad-02a7-453a-bac9-e08c2b0c88dc None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:18.699351 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23eaad-02a7-453a-bac9-e08c2b0c88dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:18.721620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fac205-fc29-43c7-971d-202d63a6b704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:18.722447 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fac205-fc29-43c7-971d-202d63a6b704 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:18.722722 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fac205-fc29-43c7-971d-202d63a6b704 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:18.722997 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fac205-fc29-43c7-971d-202d63a6b704 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:18.734724 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fac205-fc29-43c7-971d-202d63a6b704 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:18.736182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fac205-fc29-43c7-971d-202d63a6b704 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:18.757274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-e9fac205-fc29-43c7-971d-202d63a6b704 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:18.757274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:18.757274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:18.757274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:18.757274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:18.757274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:18.757274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:18.757274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:18.757274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:18.757274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:18.757274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:18.757274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:18.757274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:18.757274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:18.757274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:18.758238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:18.758238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:18.758238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:18.758238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:18.758238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:18.758238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:18.758238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:18.758238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:18.758238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:18.758238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:18.758238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:18.758238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:18.758238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:18.758238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:18.758238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:18.758238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:18.758238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:18.759144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:18.759144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:18.759144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:18.759144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:18.759981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 281/568] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:18.761330 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9ffec-a0c7-474e-8543-ac4eed2d17eb None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:18.761686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9ffec-a0c7-474e-8543-ac4eed2d17eb None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:18.762033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9ffec-a0c7-474e-8543-ac4eed2d17eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:18.776175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23eaad-02a7-453a-bac9-e08c2b0c88dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:18.777619 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23eaad-02a7-453a-bac9-e08c2b0c88dc None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:18.779460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-32c9ffec-a0c7-474e-8543-ac4eed2d17eb None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:18.780637 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-32c9ffec-a0c7-474e-8543-ac4eed2d17eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:18.847867 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 282/569] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:49:18.849739 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cee9c6-801e-4f9a-abd6-07cbe320c4da None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:18.850027 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cee9c6-801e-4f9a-abd6-07cbe320c4da None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:18.850327 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cee9c6-801e-4f9a-abd6-07cbe320c4da None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:18.855726 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 283/570] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:18.857026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cee9c6-801e-4f9a-abd6-07cbe320c4da None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:18.885634 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 288/571] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:49:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:49:18.887261 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa24b0-3d26-4c6a-b2a7-2fb0b150d83d None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:18.887546 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa24b0-3d26-4c6a-b2a7-2fb0b150d83d None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:18.887828 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa24b0-3d26-4c6a-b2a7-2fb0b150d83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:18.904315 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-84fa24b0-3d26-4c6a-b2a7-2fb0b150d83d None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:18.905502 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-84fa24b0-3d26-4c6a-b2a7-2fb0b150d83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:18.970631 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cee9c6-801e-4f9a-abd6-07cbe320c4da None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:18.982294 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 289/572] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:49:18.982294 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa24b0-3d26-4c6a-b2a7-2fb0b150d83d None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:19.022339 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c6e74-3af4-475d-8e6a-edd705a82295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:19.023003 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c6e74-3af4-475d-8e6a-edd705a82295 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:19.023284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c6e74-3af4-475d-8e6a-edd705a82295 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:19.023557 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c6e74-3af4-475d-8e6a-edd705a82295 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:19.024701 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c6e74-3af4-475d-8e6a-edd705a82295 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:19.025297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c6e74-3af4-475d-8e6a-edd705a82295 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:19.027198 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c6e74-3af4-475d-8e6a-edd705a82295 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:19.046489 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 284/573] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:19.046489 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c6e74-3af4-475d-8e6a-edd705a82295 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:19.071534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa24b0-3d26-4c6a-b2a7-2fb0b150d83d None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:19.106193 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc3b22-9cb5-4708-9933-93289897a82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:19.106987 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc3b22-9cb5-4708-9933-93289897a82a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:19.107269 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc3b22-9cb5-4708-9933-93289897a82a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:19.107543 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc3b22-9cb5-4708-9933-93289897a82a None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:19.117683 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc3b22-9cb5-4708-9933-93289897a82a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:19.125372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-37dc3b22-9cb5-4708-9933-93289897a82a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:19.125372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:19.125372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:19.125372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:19.125372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:19.125372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:19.125372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:19.125372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:19.125372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:19.125372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:19.125372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:19.125372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:19.125372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:19.125372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:19.125372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:19.125372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:19.126823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:19.126823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:19.126823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:19.126823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:19.126823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:19.126823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:19.126823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:19.126823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:19.126823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:19.126823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:19.126823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:19.126823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:19.126823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:19.126823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:19.126823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:19.126823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:19.127650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:19.127650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:19.127650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:19.127650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:19.127650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:19.127650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:19.127650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:19.127650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:19.127650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:19.127650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:19.127650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:19.127650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:19.130953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 290/574] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 9 22:49:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:19.131811 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc3b22-9cb5-4708-9933-93289897a82a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:19.153013 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c6e74-3af4-475d-8e6a-edd705a82295 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:19.186304 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a28004-16d8-40be-a37e-23291ad1f0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:19.187097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a28004-16d8-40be-a37e-23291ad1f0f3 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:19.187376 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a28004-16d8-40be-a37e-23291ad1f0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:19.187647 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a28004-16d8-40be-a37e-23291ad1f0f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:19.198719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a28004-16d8-40be-a37e-23291ad1f0f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:19.200170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a28004-16d8-40be-a37e-23291ad1f0f3 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:19.217211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-b4a28004-16d8-40be-a37e-23291ad1f0f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:19.217211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:19.217211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:19.217211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:19.217211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:19.217211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:19.217211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:19.217211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:19.217211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:19.217211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:19.217211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:19.217211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:19.217211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:19.217211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:19.217211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:19.218209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:19.218209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:19.218209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:19.218209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:19.218209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:19.218209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:19.218209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:19.218209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:19.218209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:19.218209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:19.218209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:19.218209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:19.218209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:19.218209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:19.218209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:19.218209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:19.218209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:19.219193 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:19.219193 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:19.219193 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:19.219193 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:19.219193 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 285/575] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:19.219193 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a28004-16d8-40be-a37e-23291ad1f0f3 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:19.245822 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc3b22-9cb5-4708-9933-93289897a82a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:19.286815 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e547b-145a-4579-8245-8f0767c648ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:19.287612 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e547b-145a-4579-8245-8f0767c648ac None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:19.287886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e547b-145a-4579-8245-8f0767c648ac None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:19.288154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e547b-145a-4579-8245-8f0767c648ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:19.297941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e547b-145a-4579-8245-8f0767c648ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:19.306088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-603e547b-145a-4579-8245-8f0767c648ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:19.306088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:19.306088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:19.306088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:19.306088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:19.306088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:19.306088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:19.306088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:19.306088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:19.306088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:19.306088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:19.306088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:19.306088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:19.306088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:19.306088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:19.306088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:19.307024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:19.307024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:19.307024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:19.307024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:19.307024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:19.307024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:19.307024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:19.307024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:19.307024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:19.307024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:19.307024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:19.307024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:19.307024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:19.307024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:19.307024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:19.307024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:19.307885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:19.307885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:19.307885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:19.307885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:19.307885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:19.307885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:19.307885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:19.307885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:19.307885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:19.307885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:19.307885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:19.307885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:19.307885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 291/576] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 9 22:49:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:19.308571 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e547b-145a-4579-8245-8f0767c648ac None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:19.324289 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a28004-16d8-40be-a37e-23291ad1f0f3 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:19.356536 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657be651-17b2-426e-bb46-1ba399036425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:19.357354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657be651-17b2-426e-bb46-1ba399036425 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:19.357631 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657be651-17b2-426e-bb46-1ba399036425 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:19.357959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657be651-17b2-426e-bb46-1ba399036425 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:19.366481 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657be651-17b2-426e-bb46-1ba399036425 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:19.368016 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657be651-17b2-426e-bb46-1ba399036425 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:19.382522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-657be651-17b2-426e-bb46-1ba399036425 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 22:49:19.382522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:19.382522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:19.382522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:19.382522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:19.382522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:19.382522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:19.382522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:19.382522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:19.382522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:19.382522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:19.382522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:19.382522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:19.382522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:19.382522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:19.382522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:19.383490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:19.383490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:19.383490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:19.383490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:19.383490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:19.383490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:19.383490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:19.383490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:19.383490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:19.383490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:19.383490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:19.383490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:19.383490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:19.383490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:19.383490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:19.383490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:19.383490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:19.383490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:19.384471 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:19.384471 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:19.384471 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:19.384471 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:19.384471 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:19.384471 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:19.384471 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:19.384471 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:19.384471 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 22:49:19.384471 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:19.384471 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 286/577] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:49:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:19.385058 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657be651-17b2-426e-bb46-1ba399036425 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:19.408375 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e547b-145a-4579-8245-8f0767c648ac None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:19.441581 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df59e7c-0f91-4a1a-8af8-7ff19b4acda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:19.442270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df59e7c-0f91-4a1a-8af8-7ff19b4acda6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:19.442550 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df59e7c-0f91-4a1a-8af8-7ff19b4acda6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:19.442892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df59e7c-0f91-4a1a-8af8-7ff19b4acda6 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:19.443633 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df59e7c-0f91-4a1a-8af8-7ff19b4acda6 None admin] RBAC: Adding query filter params (name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:19.443938 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df59e7c-0f91-4a1a-8af8-7ff19b4acda6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:19.445304 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df59e7c-0f91-4a1a-8af8-7ff19b4acda6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:19.457287 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 292/578] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 9 22:49:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:19.457287 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df59e7c-0f91-4a1a-8af8-7ff19b4acda6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:19.490889 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657be651-17b2-426e-bb46-1ba399036425 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:19.522903 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf108425-ef26-4292-8b2a-93ded87fd01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:19.523566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf108425-ef26-4292-8b2a-93ded87fd01a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:19.523847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf108425-ef26-4292-8b2a-93ded87fd01a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:19.524121 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf108425-ef26-4292-8b2a-93ded87fd01a None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:19.524831 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf108425-ef26-4292-8b2a-93ded87fd01a None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:19.525134 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf108425-ef26-4292-8b2a-93ded87fd01a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:19.526526 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf108425-ef26-4292-8b2a-93ded87fd01a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:19.542769 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 287/579] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:19.542769 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf108425-ef26-4292-8b2a-93ded87fd01a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:19.551899 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df59e7c-0f91-4a1a-8af8-7ff19b4acda6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:19.584952 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dfda7e-a68d-47ba-b1fe-ae4c3ef091dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:19.585617 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dfda7e-a68d-47ba-b1fe-ae4c3ef091dc None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:19.585886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dfda7e-a68d-47ba-b1fe-ae4c3ef091dc None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:19.586184 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dfda7e-a68d-47ba-b1fe-ae4c3ef091dc None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:19.586953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dfda7e-a68d-47ba-b1fe-ae4c3ef091dc None admin] RBAC: Adding query filter params (name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:19.587209 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dfda7e-a68d-47ba-b1fe-ae4c3ef091dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:19.588555 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dfda7e-a68d-47ba-b1fe-ae4c3ef091dc None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:19.596336 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 293/580] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 9 22:49:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:19.597349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dfda7e-a68d-47ba-b1fe-ae4c3ef091dc None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:19.647520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf108425-ef26-4292-8b2a-93ded87fd01a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:19.682167 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7aac3-44c3-40c4-8eee-e401afd4ea88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:19.682901 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7aac3-44c3-40c4-8eee-e401afd4ea88 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:19.683189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7aac3-44c3-40c4-8eee-e401afd4ea88 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:19.683461 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7aac3-44c3-40c4-8eee-e401afd4ea88 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:19.684295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb7aac3-44c3-40c4-8eee-e401afd4ea88 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:19.684604 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb7aac3-44c3-40c4-8eee-e401afd4ea88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:19.686024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb7aac3-44c3-40c4-8eee-e401afd4ea88 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:19.699571 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dfda7e-a68d-47ba-b1fe-ae4c3ef091dc None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:19.702978 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 288/581] 10.222.0.23 () {56 vars in 1137 bytes} [Mon May 9 22:49:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:19.704008 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7aac3-44c3-40c4-8eee-e401afd4ea88 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:19.735003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4f17c-e743-41d4-8ac8-caaa03cca18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:19.735792 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4f17c-e743-41d4-8ac8-caaa03cca18a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:19.736070 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4f17c-e743-41d4-8ac8-caaa03cca18a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:19.736350 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4f17c-e743-41d4-8ac8-caaa03cca18a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:19.747307 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc4f17c-e743-41d4-8ac8-caaa03cca18a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:19.748746 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc4f17c-e743-41d4-8ac8-caaa03cca18a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:19.765525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-1bc4f17c-e743-41d4-8ac8-caaa03cca18a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:19.765525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:19.765525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:19.765525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:19.765525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:19.765525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:19.765525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:19.765525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:19.765525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:19.765525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:19.765525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:19.765525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:19.765525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:19.765525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:19.765525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:19.766657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:19.766657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:19.766657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:19.766657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:19.766657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:19.766657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:19.766657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:19.766657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:19.766657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:19.766657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:19.766657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:19.766657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:19.766657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:19.766657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:19.766657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:19.766657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:19.766657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:19.767731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:19.767731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:19.767731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:19.767731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:19.768336 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 294/582] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:19.769113 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4f17c-e743-41d4-8ac8-caaa03cca18a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:19.813854 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7aac3-44c3-40c4-8eee-e401afd4ea88 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:19.846555 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26d9f5-ced4-4f0a-a9b0-475f0c6821db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:19.847328 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26d9f5-ced4-4f0a-a9b0-475f0c6821db None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:19.847603 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26d9f5-ced4-4f0a-a9b0-475f0c6821db None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:19.847870 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26d9f5-ced4-4f0a-a9b0-475f0c6821db None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:19.854928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d26d9f5-ced4-4f0a-a9b0-475f0c6821db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:19.856358 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d26d9f5-ced4-4f0a-a9b0-475f0c6821db None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:19.870367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-5d26d9f5-ced4-4f0a-a9b0-475f0c6821db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:19.870367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:19.870367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:19.870367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:19.870367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:19.870367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:19.870367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:19.870367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:19.870367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:19.870367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:19.870367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:19.870367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:19.870367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:19.870367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:19.870367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:19.871297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:19.871297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:19.871297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:19.871297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:19.871297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:19.871297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:19.871297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:19.871297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:19.871297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:19.871297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:19.871297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:19.871297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:19.871297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:19.871297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:19.871297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:19.871297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:19.871297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:19.872212 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:19.872212 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:19.872212 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:19.872212 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:19.872212 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:19.872212 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:19.872212 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:19.872212 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:19.872212 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 289/583] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:19.872716 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26d9f5-ced4-4f0a-a9b0-475f0c6821db None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:19.879682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4f17c-e743-41d4-8ac8-caaa03cca18a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:19.913276 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13434fe-f91c-4ab0-8ba8-d89727682822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:19.914059 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13434fe-f91c-4ab0-8ba8-d89727682822 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:19.914472 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13434fe-f91c-4ab0-8ba8-d89727682822 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:19.914753 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13434fe-f91c-4ab0-8ba8-d89727682822 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:19.924850 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13434fe-f91c-4ab0-8ba8-d89727682822 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:19.926317 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13434fe-f91c-4ab0-8ba8-d89727682822 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:19.943411 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-a13434fe-f91c-4ab0-8ba8-d89727682822 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:19.943411 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:19.943411 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:19.943411 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:19.943411 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:19.943411 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:19.943411 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:19.943411 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:19.943411 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:19.943411 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:19.943411 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:19.943411 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:19.943411 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:19.943411 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:19.943411 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:19.944293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:19.944293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:19.944293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:19.944293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:19.944293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:19.944293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:19.944293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:19.944293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:19.944293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:19.944293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:19.944293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:19.944293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:19.944293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:19.944293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:19.944293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:19.944293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:19.944293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:19.945184 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:19.945184 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:19.945184 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:19.945184 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:19.945184 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 295/584] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:19.945459 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13434fe-f91c-4ab0-8ba8-d89727682822 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:19.975888 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26d9f5-ced4-4f0a-a9b0-475f0c6821db None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:20.007081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ef594-cb26-45d8-af35-0473bd5e1ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:20.007861 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ef594-cb26-45d8-af35-0473bd5e1ef2 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:20.008145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ef594-cb26-45d8-af35-0473bd5e1ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:20.008413 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ef594-cb26-45d8-af35-0473bd5e1ef2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:20.018643 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ef594-cb26-45d8-af35-0473bd5e1ef2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:20.020150 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ef594-cb26-45d8-af35-0473bd5e1ef2 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:20.037951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-363ef594-cb26-45d8-af35-0473bd5e1ef2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:20.037951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:20.037951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:20.037951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:20.037951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:20.037951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:20.037951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:20.037951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:20.037951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:20.037951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:20.037951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:20.037951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:20.037951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:20.037951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:20.037951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:20.038995 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:20.038995 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:20.038995 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:20.038995 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:20.038995 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:20.038995 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:20.038995 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:20.038995 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:20.038995 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:20.038995 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:20.038995 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:20.038995 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:20.038995 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:20.038995 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:20.038995 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:20.038995 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:20.038995 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:20.040426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:20.040426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:20.040426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:20.040426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:20.040426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 290/585] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:20.040426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ef594-cb26-45d8-af35-0473bd5e1ef2 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:20.043932 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13434fe-f91c-4ab0-8ba8-d89727682822 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:20.077182 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3494c1-4315-44af-ae95-ae44ad2b2a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:20.077866 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3494c1-4315-44af-ae95-ae44ad2b2a5c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:20.078238 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3494c1-4315-44af-ae95-ae44ad2b2a5c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:20.078571 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3494c1-4315-44af-ae95-ae44ad2b2a5c None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:20.079344 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3494c1-4315-44af-ae95-ae44ad2b2a5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:20.079734 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3494c1-4315-44af-ae95-ae44ad2b2a5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:20.081133 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3494c1-4315-44af-ae95-ae44ad2b2a5c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:20.097197 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 296/586] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:20.097197 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3494c1-4315-44af-ae95-ae44ad2b2a5c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:20.140047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ef594-cb26-45d8-af35-0473bd5e1ef2 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:20.172927 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1f2cc-0c84-4594-80fe-7ac46b0c8649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:20.173589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1f2cc-0c84-4594-80fe-7ac46b0c8649 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:20.173872 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1f2cc-0c84-4594-80fe-7ac46b0c8649 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:20.174162 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1f2cc-0c84-4594-80fe-7ac46b0c8649 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:20.174942 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1f2cc-0c84-4594-80fe-7ac46b0c8649 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:20.175245 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1f2cc-0c84-4594-80fe-7ac46b0c8649 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:20.176668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1f2cc-0c84-4594-80fe-7ac46b0c8649 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:20.193238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 291/587] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:20.193238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1f2cc-0c84-4594-80fe-7ac46b0c8649 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:20.193497 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3494c1-4315-44af-ae95-ae44ad2b2a5c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:20.226684 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c1a78-de92-4b1f-b00d-56b594c6394a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:20.230155 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c1a78-de92-4b1f-b00d-56b594c6394a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:20.230155 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c1a78-de92-4b1f-b00d-56b594c6394a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:20.230155 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c1a78-de92-4b1f-b00d-56b594c6394a None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:20.230155 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c1a78-de92-4b1f-b00d-56b594c6394a None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:20.230155 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c1a78-de92-4b1f-b00d-56b594c6394a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:20.230544 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c1a78-de92-4b1f-b00d-56b594c6394a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:20.243085 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 297/588] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:20.244267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c1a78-de92-4b1f-b00d-56b594c6394a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:20.295114 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1f2cc-0c84-4594-80fe-7ac46b0c8649 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:20.328145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ea8f3-c326-4f00-9f9c-a2f601c0c1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:20.328878 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ea8f3-c326-4f00-9f9c-a2f601c0c1d8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:20.329378 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ea8f3-c326-4f00-9f9c-a2f601c0c1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:20.329606 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ea8f3-c326-4f00-9f9c-a2f601c0c1d8 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:20.330448 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ea8f3-c326-4f00-9f9c-a2f601c0c1d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:20.330844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ea8f3-c326-4f00-9f9c-a2f601c0c1d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:20.332296 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ea8f3-c326-4f00-9f9c-a2f601c0c1d8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:20.344110 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 292/589] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:20.344261 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c1a78-de92-4b1f-b00d-56b594c6394a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:20.345185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ea8f3-c326-4f00-9f9c-a2f601c0c1d8 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:20.377021 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700b137-a2d3-4474-8206-effe5f01ad63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:20.377909 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700b137-a2d3-4474-8206-effe5f01ad63 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:20.378213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700b137-a2d3-4474-8206-effe5f01ad63 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:20.378486 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700b137-a2d3-4474-8206-effe5f01ad63 None admin] PATH_INFO: `/v3/users/glance` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:20.386718 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700b137-a2d3-4474-8206-effe5f01ad63 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:20.388302 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700b137-a2d3-4474-8206-effe5f01ad63 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:20.402993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-a700b137-a2d3-4474-8206-effe5f01ad63 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 09 22:49:20.402993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:20.402993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:20.402993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:20.402993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:20.402993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:20.402993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:20.402993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:20.402993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:20.402993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:20.402993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:20.402993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:20.402993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:20.402993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:20.402993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:20.402993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:20.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:20.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:20.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:20.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:20.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:20.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:20.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:20.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:20.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:20.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:20.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:20.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:20.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:20.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:20.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:20.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:20.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:20.404109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:20.405251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:20.405251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:20.405251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:20.405251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:20.405251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:20.405251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:20.405251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:20.405251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:20.405251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 09 22:49:20.405251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:20.411597 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 298/590] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:49:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:20.412578 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700b137-a2d3-4474-8206-effe5f01ad63 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:20.440010 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ea8f3-c326-4f00-9f9c-a2f601c0c1d8 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:20.472855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcef9ff-318c-4f4e-8fc2-87b0b1e2bea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:20.473502 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcef9ff-318c-4f4e-8fc2-87b0b1e2bea9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:20.473778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcef9ff-318c-4f4e-8fc2-87b0b1e2bea9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:20.474075 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcef9ff-318c-4f4e-8fc2-87b0b1e2bea9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:20.474918 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcef9ff-318c-4f4e-8fc2-87b0b1e2bea9 None admin] RBAC: Adding query filter params (role.id=44847d50e6cb4bc7938c59b4e2d99833, scope.project.id=0500e378ea374ca692e230bbd8781318, user.id=8cb6c294ce1842b68a6bc73ac381ed6b) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:20.475224 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcef9ff-318c-4f4e-8fc2-87b0b1e2bea9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:20.476634 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcef9ff-318c-4f4e-8fc2-87b0b1e2bea9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:20.482799 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 293/591] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:49:20 2022] GET /identity/v3/role_assignments?user.id=8cb6c294ce1842b68a6bc73ac381ed6b&scope.project.id=0500e378ea374ca692e230bbd8781318&role.id=44847d50e6cb4bc7938c59b4e2d99833 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:20.483840 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcef9ff-318c-4f4e-8fc2-87b0b1e2bea9 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:20.512955 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700b137-a2d3-4474-8206-effe5f01ad63 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:20.549418 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec690dca-b99d-4e24-92e2-1f1a6539f9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:20.550261 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec690dca-b99d-4e24-92e2-1f1a6539f9e7 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:20.550543 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec690dca-b99d-4e24-92e2-1f1a6539f9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:20.550818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec690dca-b99d-4e24-92e2-1f1a6539f9e7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:20.559517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec690dca-b99d-4e24-92e2-1f1a6539f9e7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:20.560988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec690dca-b99d-4e24-92e2-1f1a6539f9e7 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:20.577097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-ec690dca-b99d-4e24-92e2-1f1a6539f9e7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 09 22:49:20.577097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:20.577097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:20.577097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:20.577097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:20.577097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:20.577097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:20.577097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:20.577097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:20.577097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:20.577097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:20.577097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:20.577097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:20.577097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:20.577097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:20.577097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:20.578106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:20.578106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:20.578106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:20.578106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:20.578106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:20.578106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:20.578106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:20.578106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:20.578106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:20.578106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:20.578106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:20.578106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:20.578106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:20.578106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:20.578106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:20.578106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:20.578106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:20.578106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:20.579118 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:20.579118 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:20.579118 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:20.579118 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:20.579118 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:20.579118 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:20.579118 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:20.579118 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:20.579118 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 09 22:49:20.579118 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:20.584017 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 299/592] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 9 22:49:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:20.584739 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec690dca-b99d-4e24-92e2-1f1a6539f9e7 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:20.610901 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcef9ff-318c-4f4e-8fc2-87b0b1e2bea9 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:20.653872 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74c521-47fe-4661-a122-97a7922ea569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:20.655216 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74c521-47fe-4661-a122-97a7922ea569 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:20.655730 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74c521-47fe-4661-a122-97a7922ea569 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:20.656233 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74c521-47fe-4661-a122-97a7922ea569 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:20.664047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74c521-47fe-4661-a122-97a7922ea569 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:20.665867 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74c521-47fe-4661-a122-97a7922ea569 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:20.681790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-5b74c521-47fe-4661-a122-97a7922ea569 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:20.681790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:20.681790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:20.681790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:20.681790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:20.681790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:20.681790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:20.681790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:20.681790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:20.681790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:20.681790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:20.681790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:20.681790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:20.681790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:20.681790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:20.682776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:20.682776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:20.682776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:20.682776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:20.682776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:20.682776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:20.682776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:20.682776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:20.682776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:20.682776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:20.682776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:20.682776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:20.682776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:20.682776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:20.682776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:20.682776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:20.682776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:20.683662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:20.683662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:20.683662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:20.683662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:20.683662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:20.683662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:20.683662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:20.683662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:20.686078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 294/593] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:20.687175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74c521-47fe-4661-a122-97a7922ea569 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:20.701935 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec690dca-b99d-4e24-92e2-1f1a6539f9e7 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:20.742084 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b5f78-65f6-46bd-8d7f-9e916acb9d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:20.743510 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b5f78-65f6-46bd-8d7f-9e916acb9d87 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:20.744042 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b5f78-65f6-46bd-8d7f-9e916acb9d87 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:20.744489 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b5f78-65f6-46bd-8d7f-9e916acb9d87 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:20.745644 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b5f78-65f6-46bd-8d7f-9e916acb9d87 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:20.746403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b5f78-65f6-46bd-8d7f-9e916acb9d87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:20.747989 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b5f78-65f6-46bd-8d7f-9e916acb9d87 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:20.772286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 300/594] 10.222.0.23 () {56 vars in 1137 bytes} [Mon May 9 22:49:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:20.773399 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b5f78-65f6-46bd-8d7f-9e916acb9d87 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:20.798972 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74c521-47fe-4661-a122-97a7922ea569 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:20.833614 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6420faa-c292-417e-81d0-b0ece4be1e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:20.834265 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6420faa-c292-417e-81d0-b0ece4be1e68 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:20.834547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6420faa-c292-417e-81d0-b0ece4be1e68 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:20.834821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6420faa-c292-417e-81d0-b0ece4be1e68 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:20.835645 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6420faa-c292-417e-81d0-b0ece4be1e68 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:20.835951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6420faa-c292-417e-81d0-b0ece4be1e68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:20.837357 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6420faa-c292-417e-81d0-b0ece4be1e68 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:20.854729 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 295/595] 10.222.0.23 () {56 vars in 1139 bytes} [Mon May 9 22:49:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:20.859188 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6420faa-c292-417e-81d0-b0ece4be1e68 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:20.877373 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b5f78-65f6-46bd-8d7f-9e916acb9d87 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:20.912197 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb63c84-392c-4a8a-92b4-6a761a807697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:20.913085 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb63c84-392c-4a8a-92b4-6a761a807697 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:20.913567 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb63c84-392c-4a8a-92b4-6a761a807697 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:20.913987 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb63c84-392c-4a8a-92b4-6a761a807697 None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:20.915126 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb63c84-392c-4a8a-92b4-6a761a807697 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:20.915577 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb63c84-392c-4a8a-92b4-6a761a807697 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:20.917177 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb63c84-392c-4a8a-92b4-6a761a807697 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:20.929593 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 301/596] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:20.931023 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744cc8a5-110c-4bd9-8d84-18bbef5a4f83 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:20.931486 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744cc8a5-110c-4bd9-8d84-18bbef5a4f83 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:20.931760 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744cc8a5-110c-4bd9-8d84-18bbef5a4f83 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:20.934935 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 302/597] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:20.934935 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744cc8a5-110c-4bd9-8d84-18bbef5a4f83 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:20.961412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6420faa-c292-417e-81d0-b0ece4be1e68 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:20.994339 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c7a85-c8b9-4ab2-959c-9c769b725ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:20.995130 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c7a85-c8b9-4ab2-959c-9c769b725ff7 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:20.995412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c7a85-c8b9-4ab2-959c-9c769b725ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:20.995689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c7a85-c8b9-4ab2-959c-9c769b725ff7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:21.006333 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c7a85-c8b9-4ab2-959c-9c769b725ff7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:21.007874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c7a85-c8b9-4ab2-959c-9c769b725ff7 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:21.028049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-825c7a85-c8b9-4ab2-959c-9c769b725ff7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:21.028049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:21.028049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:21.028049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:21.028049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:21.028049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:21.028049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:21.028049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:21.028049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:21.028049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:21.028049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:21.028049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:21.028049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:21.028049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:21.028049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:21.028977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:21.028977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:21.028977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:21.028977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:21.028977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:21.028977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:21.028977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:21.028977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:21.028977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:21.028977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:21.028977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:21.028977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:21.028977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:21.028977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:21.028977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:21.028977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:21.028977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:21.029866 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:21.029866 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:21.029866 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:21.029866 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:21.036379 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 296/598] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:21.036379 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c7a85-c8b9-4ab2-959c-9c769b725ff7 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:21.037572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744cc8a5-110c-4bd9-8d84-18bbef5a4f83 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:21.070735 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b363d4-eb72-4dbc-93e5-30925721b6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:21.071532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b363d4-eb72-4dbc-93e5-30925721b6de None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:21.071805 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b363d4-eb72-4dbc-93e5-30925721b6de None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:21.072077 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b363d4-eb72-4dbc-93e5-30925721b6de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:21.082532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b363d4-eb72-4dbc-93e5-30925721b6de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:21.084595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b363d4-eb72-4dbc-93e5-30925721b6de None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:21.101300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-d8b363d4-eb72-4dbc-93e5-30925721b6de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:21.101300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:21.101300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:21.101300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:21.101300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:21.101300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:21.101300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:21.101300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:21.101300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:21.101300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:21.101300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:21.101300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:21.101300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:21.101300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:21.101300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:21.102380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:21.102380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:21.102380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:21.102380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:21.102380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:21.102380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:21.102380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:21.102380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:21.102380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:21.102380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:21.102380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:21.102380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:21.102380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:21.102380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:21.102380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:21.102380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:21.102380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:21.107785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:21.107785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:21.107785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:21.107785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:21.107785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 303/599] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:21.107785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d2147-f893-4b3c-8378-643eb4dfd3fb None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:21.107785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d2147-f893-4b3c-8378-643eb4dfd3fb None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:21.107785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d2147-f893-4b3c-8378-643eb4dfd3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:21.122301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-7c4d2147-f893-4b3c-8378-643eb4dfd3fb None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:21.123559 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7c4d2147-f893-4b3c-8378-643eb4dfd3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:21.141799 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c7a85-c8b9-4ab2-959c-9c769b725ff7 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:21.174588 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cee88-49d7-4430-8947-683e763e2b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:21.175558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cee88-49d7-4430-8947-683e763e2b50 None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:21.175906 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cee88-49d7-4430-8947-683e763e2b50 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:21.176248 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cee88-49d7-4430-8947-683e763e2b50 None admin] PATH_INFO: `/v3/projects/0500e378ea374ca692e230bbd8781318/users/5fe668a2cee040c7a56d1f8685744cd1/roles/44847d50e6cb4bc7938c59b4e2d99833` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:21.190485 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966cee88-49d7-4430-8947-683e763e2b50 None admin] RBAC: Authorizing `identity:create_grant(project_id=0500e378ea374ca692e230bbd8781318, user_id=5fe668a2cee040c7a56d1f8685744cd1, role_id=44847d50e6cb4bc7938c59b4e2d99833)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:21.191995 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966cee88-49d7-4430-8947-683e763e2b50 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:21.195841 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 304/600] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:49:21.195841 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d2147-f893-4b3c-8378-643eb4dfd3fb None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:21.212238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 297/601] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:49:21 2022] PUT /identity/v3/projects/0500e378ea374ca692e230bbd8781318/users/5fe668a2cee040c7a56d1f8685744cd1/roles/44847d50e6cb4bc7938c59b4e2d99833 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:49:21.213042 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cee88-49d7-4430-8947-683e763e2b50 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:21.305907 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d2147-f893-4b3c-8378-643eb4dfd3fb None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:21.318597 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cee88-49d7-4430-8947-683e763e2b50 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:21.343666 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c8d58-c35c-415c-b259-eef1506a4537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:21.344256 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c8d58-c35c-415c-b259-eef1506a4537 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:21.344538 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c8d58-c35c-415c-b259-eef1506a4537 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:21.344813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c8d58-c35c-415c-b259-eef1506a4537 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:21.345544 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c8d58-c35c-415c-b259-eef1506a4537 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:21.345847 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c8d58-c35c-415c-b259-eef1506a4537 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:21.347269 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c8d58-c35c-415c-b259-eef1506a4537 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:21.357070 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6661c-1073-4d79-b5ac-bd730455f119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:21.357902 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6661c-1073-4d79-b5ac-bd730455f119 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:21.358313 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6661c-1073-4d79-b5ac-bd730455f119 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:21.358686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6661c-1073-4d79-b5ac-bd730455f119 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:21.359775 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6661c-1073-4d79-b5ac-bd730455f119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:21.360038 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 305/602] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:21.360284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6661c-1073-4d79-b5ac-bd730455f119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:21.360885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c8d58-c35c-415c-b259-eef1506a4537 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:21.361838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6661c-1073-4d79-b5ac-bd730455f119 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:21.374623 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 298/603] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:21.375686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6661c-1073-4d79-b5ac-bd730455f119 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:21.468082 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c8d58-c35c-415c-b259-eef1506a4537 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:21.480369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6661c-1073-4d79-b5ac-bd730455f119 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:21.503402 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4abe2-4570-45fe-bc2d-f0f958430821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:21.504370 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4abe2-4570-45fe-bc2d-f0f958430821 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:21.504747 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4abe2-4570-45fe-bc2d-f0f958430821 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:21.505116 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4abe2-4570-45fe-bc2d-f0f958430821 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:21.516090 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4abe2-4570-45fe-bc2d-f0f958430821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:21.518086 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba067f2c-c330-4b01-a335-ebf06f739ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:21.519079 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba067f2c-c330-4b01-a335-ebf06f739ffe None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:21.519503 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba067f2c-c330-4b01-a335-ebf06f739ffe None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:21.519918 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba067f2c-c330-4b01-a335-ebf06f739ffe None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:21.524280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-39e4abe2-4570-45fe-bc2d-f0f958430821 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:21.524280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:21.524280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:21.524280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:21.524280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:21.524280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:21.524280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:21.524280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:21.524280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:21.524280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:21.524280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:21.524280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:21.524280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:21.524280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:21.524280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:21.524280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:21.525404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:21.525404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:21.525404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:21.525404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:21.525404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:21.525404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:21.525404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:21.525404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:21.525404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:21.525404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:21.525404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:21.525404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:21.525404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:21.525404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:21.525404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:21.525404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:21.526471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:21.526471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:21.526471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:21.526471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:21.526471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:21.526471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:21.526471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:21.526471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:21.526471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:21.526471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:21.526471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:21.526471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:21.528257 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 306/604] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 9 22:49:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:21.529177 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4abe2-4570-45fe-bc2d-f0f958430821 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:21.530711 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba067f2c-c330-4b01-a335-ebf06f739ffe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:21.532352 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba067f2c-c330-4b01-a335-ebf06f739ffe None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:21.547037 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-ba067f2c-c330-4b01-a335-ebf06f739ffe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:21.547037 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:21.547037 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:21.547037 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:21.547037 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:21.547037 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:21.547037 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:21.547037 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:21.547037 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:21.547037 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:21.547037 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:21.547037 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:21.547037 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:21.547037 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:21.547037 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:21.547953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:21.547953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:21.547953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:21.547953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:21.547953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:21.547953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:21.547953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:21.547953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:21.547953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:21.547953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:21.547953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:21.547953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:21.547953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:21.547953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:21.547953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:21.547953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:21.547953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:21.548837 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:21.548837 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:21.548837 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:21.548837 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:21.548837 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:21.548837 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:21.548837 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:21.548837 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:21.551459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 299/605] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:21.552944 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba067f2c-c330-4b01-a335-ebf06f739ffe None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:21.633389 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4abe2-4570-45fe-bc2d-f0f958430821 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:21.666412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba067f2c-c330-4b01-a335-ebf06f739ffe None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:21.672419 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9157719-2bee-43a9-90e5-4e76fd017fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:21.673302 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9157719-2bee-43a9-90e5-4e76fd017fef None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:21.673673 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9157719-2bee-43a9-90e5-4e76fd017fef None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:21.674015 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9157719-2bee-43a9-90e5-4e76fd017fef None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:21.681736 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9157719-2bee-43a9-90e5-4e76fd017fef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:21.683347 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9157719-2bee-43a9-90e5-4e76fd017fef None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:21.698450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-a9157719-2bee-43a9-90e5-4e76fd017fef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:21.698450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:21.698450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:21.698450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:21.698450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:21.698450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:21.698450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:21.698450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:21.698450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:21.698450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:21.698450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:21.698450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:21.698450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:21.698450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:21.698450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:21.699392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:21.699392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:21.699392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:21.699392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:21.699392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:21.699392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:21.699392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:21.699392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:21.699392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:21.699392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:21.699392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:21.699392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:21.699392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:21.699392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:21.699392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:21.699392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:21.699392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:21.700279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:21.700279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:21.700279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:21.700279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:21.700279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:21.700279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:21.700279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:21.700279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:21.700279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 307/606] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:21.700772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9157719-2bee-43a9-90e5-4e76fd017fef None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:21.761318 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc3668-c626-4e84-84e4-a21ef7a24324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:21.764864 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc3668-c626-4e84-84e4-a21ef7a24324 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:21.764864 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc3668-c626-4e84-84e4-a21ef7a24324 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:21.764864 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc3668-c626-4e84-84e4-a21ef7a24324 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:21.764864 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc3668-c626-4e84-84e4-a21ef7a24324 None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:21.764864 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc3668-c626-4e84-84e4-a21ef7a24324 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:21.764864 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc3668-c626-4e84-84e4-a21ef7a24324 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:21.783004 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 300/607] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 9 22:49:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:21.787148 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc3668-c626-4e84-84e4-a21ef7a24324 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:21.897893 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9157719-2bee-43a9-90e5-4e76fd017fef None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:21.939942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b82419-8ac9-4fbd-9778-7ac95b2afd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:21.940605 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b82419-8ac9-4fbd-9778-7ac95b2afd08 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:21.940883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b82419-8ac9-4fbd-9778-7ac95b2afd08 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:21.941156 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b82419-8ac9-4fbd-9778-7ac95b2afd08 None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:21.944089 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b82419-8ac9-4fbd-9778-7ac95b2afd08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:21.945120 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b82419-8ac9-4fbd-9778-7ac95b2afd08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:21.948404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b82419-8ac9-4fbd-9778-7ac95b2afd08 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:21.980436 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc3668-c626-4e84-84e4-a21ef7a24324 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:21.980658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 308/608] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:21.981349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b82419-8ac9-4fbd-9778-7ac95b2afd08 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:22.024677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbe216-145d-4d8a-b77c-9e1ed43f88b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:22.025766 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbe216-145d-4d8a-b77c-9e1ed43f88b9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:22.026271 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbe216-145d-4d8a-b77c-9e1ed43f88b9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:22.026749 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbe216-145d-4d8a-b77c-9e1ed43f88b9 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:22.027725 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbe216-145d-4d8a-b77c-9e1ed43f88b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:22.028286 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbe216-145d-4d8a-b77c-9e1ed43f88b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:22.031241 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbe216-145d-4d8a-b77c-9e1ed43f88b9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:22.057432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 301/609] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:22.060287 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbe216-145d-4d8a-b77c-9e1ed43f88b9 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:22.155821 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b82419-8ac9-4fbd-9778-7ac95b2afd08 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:22.173889 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbe216-145d-4d8a-b77c-9e1ed43f88b9 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:22.197104 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1bab9-49c7-419e-8303-294eec5df000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:22.198042 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1bab9-49c7-419e-8303-294eec5df000 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:22.198349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1bab9-49c7-419e-8303-294eec5df000 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:22.198621 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1bab9-49c7-419e-8303-294eec5df000 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:22.210345 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f412c-a417-4f43-86ca-8907b1ff1644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:22.210530 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1bab9-49c7-419e-8303-294eec5df000 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:22.211672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f412c-a417-4f43-86ca-8907b1ff1644 None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:22.211951 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1bab9-49c7-419e-8303-294eec5df000 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:22.212573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f412c-a417-4f43-86ca-8907b1ff1644 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:22.213058 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f412c-a417-4f43-86ca-8907b1ff1644 None admin] PATH_INFO: `/v3/projects/0500e378ea374ca692e230bbd8781318/users/a81caf0558f2449cbbc8f4cd9446f363/roles/44847d50e6cb4bc7938c59b4e2d99833` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:22.228677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f412c-a417-4f43-86ca-8907b1ff1644 None admin] RBAC: Authorizing `identity:create_grant(project_id=0500e378ea374ca692e230bbd8781318, user_id=a81caf0558f2449cbbc8f4cd9446f363, role_id=44847d50e6cb4bc7938c59b4e2d99833)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:22.229523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-f6e1bab9-49c7-419e-8303-294eec5df000 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:22.229523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:22.229523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:22.229523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:22.229523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:22.229523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:22.229523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:22.229523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:22.229523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:22.229523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:22.229523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:22.229523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:22.229523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:22.229523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:22.229523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:22.230451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:22.230451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:22.230451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:22.230451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:22.230451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:22.230451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:22.230451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:22.230451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:22.230451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:22.230451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:22.230451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:22.230451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:22.230451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:22.230451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:22.230451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:22.230451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:22.230451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:22.231353 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:22.231353 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:22.231353 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:22.231353 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:22.231353 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 309/610] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:22.232062 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1bab9-49c7-419e-8303-294eec5df000 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:22.233307 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f412c-a417-4f43-86ca-8907b1ff1644 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:22.249036 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 302/611] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:49:22 2022] PUT /identity/v3/projects/0500e378ea374ca692e230bbd8781318/users/a81caf0558f2449cbbc8f4cd9446f363/roles/44847d50e6cb4bc7938c59b4e2d99833 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:49:22.250304 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f412c-a417-4f43-86ca-8907b1ff1644 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:22.341948 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1bab9-49c7-419e-8303-294eec5df000 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:22.366779 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f412c-a417-4f43-86ca-8907b1ff1644 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:22.377378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96382085-2075-4742-b519-2b21244d0ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:22.378517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96382085-2075-4742-b519-2b21244d0ff4 None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:22.378633 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96382085-2075-4742-b519-2b21244d0ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:22.378857 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96382085-2075-4742-b519-2b21244d0ff4 None admin] PATH_INFO: `/v3/projects/0500e378ea374ca692e230bbd8781318/users/96fd3f6dd9e74ba7a4ad159e43e8ae96/roles/44847d50e6cb4bc7938c59b4e2d99833` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:22.395004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96382085-2075-4742-b519-2b21244d0ff4 None admin] RBAC: Authorizing `identity:create_grant(project_id=0500e378ea374ca692e230bbd8781318, user_id=96fd3f6dd9e74ba7a4ad159e43e8ae96, role_id=44847d50e6cb4bc7938c59b4e2d99833)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:22.396482 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96382085-2075-4742-b519-2b21244d0ff4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:22.406932 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f669169-8a28-4a8f-8405-d9c85d76b43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:22.408019 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f669169-8a28-4a8f-8405-d9c85d76b43b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:22.408522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f669169-8a28-4a8f-8405-d9c85d76b43b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:22.409019 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f669169-8a28-4a8f-8405-d9c85d76b43b None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:22.410092 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f669169-8a28-4a8f-8405-d9c85d76b43b None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:22.410834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f669169-8a28-4a8f-8405-d9c85d76b43b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:22.413051 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f669169-8a28-4a8f-8405-d9c85d76b43b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:22.429687 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 303/612] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:22.438325 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 310/613] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:49:22 2022] PUT /identity/v3/projects/0500e378ea374ca692e230bbd8781318/users/96fd3f6dd9e74ba7a4ad159e43e8ae96/roles/44847d50e6cb4bc7938c59b4e2d99833 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:49:22.438527 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f669169-8a28-4a8f-8405-d9c85d76b43b None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:22.580692 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f669169-8a28-4a8f-8405-d9c85d76b43b None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:22.619410 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9aff5-aa54-47e4-a57e-d57b791ccd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:22.620218 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9aff5-aa54-47e4-a57e-d57b791ccd13 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:22.620497 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9aff5-aa54-47e4-a57e-d57b791ccd13 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:22.620772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9aff5-aa54-47e4-a57e-d57b791ccd13 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:22.639890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9aff5-aa54-47e4-a57e-d57b791ccd13 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:22.641459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9aff5-aa54-47e4-a57e-d57b791ccd13 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:22.657896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-53d9aff5-aa54-47e4-a57e-d57b791ccd13 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 22:49:22.657896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:22.657896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:22.657896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:22.657896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:22.657896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:22.657896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:22.657896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:22.657896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:22.657896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:22.657896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:22.657896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:22.657896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:22.657896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:22.657896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:22.657896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:22.659089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:22.659089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:22.659089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:22.659089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:22.659089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:22.659089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:22.659089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:22.659089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:22.659089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:22.659089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:22.659089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:22.659089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:22.659089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:22.659089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:22.659089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:22.659089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:22.659089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:22.659089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:22.660320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:22.660320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:22.660320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:22.660320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:22.660320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:22.660320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:22.660320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:22.660320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:22.660320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 22:49:22.660320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:22.661105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 304/614] 10.222.0.23 () {56 vars in 1093 bytes} [Mon May 9 22:49:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:22.668171 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96382085-2075-4742-b519-2b21244d0ff4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:22.775882 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96382085-2075-4742-b519-2b21244d0ff4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:22.810565 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2668fb3c-eb21-47fb-91ad-b65c2ffeb3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:22.811338 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668fb3c-eb21-47fb-91ad-b65c2ffeb3ae None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:22.811690 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668fb3c-eb21-47fb-91ad-b65c2ffeb3ae None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:22.812025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668fb3c-eb21-47fb-91ad-b65c2ffeb3ae None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:22.812934 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2668fb3c-eb21-47fb-91ad-b65c2ffeb3ae None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:22.813316 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2668fb3c-eb21-47fb-91ad-b65c2ffeb3ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:22.814843 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2668fb3c-eb21-47fb-91ad-b65c2ffeb3ae None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:22.833161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 311/615] 10.222.0.23 () {56 vars in 1133 bytes} [Mon May 9 22:49:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:22.839102 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9aff5-aa54-47e4-a57e-d57b791ccd13 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:22.945977 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fcad4-abe4-46af-82f4-44e8626d6701 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:22.946293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fcad4-abe4-46af-82f4-44e8626d6701 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:22.946666 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fcad4-abe4-46af-82f4-44e8626d6701 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:22.948256 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 312/616] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:49:22.954180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559db7a-e586-479c-b6cd-94c8342a85e1 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:22.954180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559db7a-e586-479c-b6cd-94c8342a85e1 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:22.954180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559db7a-e586-479c-b6cd-94c8342a85e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:22.966980 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9aff5-aa54-47e4-a57e-d57b791ccd13 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:22.978905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-7559db7a-e586-479c-b6cd-94c8342a85e1 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:22.981914 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7559db7a-e586-479c-b6cd-94c8342a85e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:23.027821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e966e3d-b8bc-4803-a846-b9faa359d340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:23.028724 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e966e3d-b8bc-4803-a846-b9faa359d340 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:23.029156 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e966e3d-b8bc-4803-a846-b9faa359d340 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:23.029579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e966e3d-b8bc-4803-a846-b9faa359d340 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:23.107850 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 313/617] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:49:23.121400 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e966e3d-b8bc-4803-a846-b9faa359d340 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:23.122888 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e966e3d-b8bc-4803-a846-b9faa359d340 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:23.148007 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559db7a-e586-479c-b6cd-94c8342a85e1 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:23.246830 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 305/618] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:49:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:49:23.255139 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e966e3d-b8bc-4803-a846-b9faa359d340 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:23.299508 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559db7a-e586-479c-b6cd-94c8342a85e1 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:23.339457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc256bbe-33b6-4df6-9645-8cee7128e59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:23.340223 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc256bbe-33b6-4df6-9645-8cee7128e59e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:23.340503 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc256bbe-33b6-4df6-9645-8cee7128e59e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:23.340769 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc256bbe-33b6-4df6-9645-8cee7128e59e None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:23.365038 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc256bbe-33b6-4df6-9645-8cee7128e59e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:23.379835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-dc256bbe-33b6-4df6-9645-8cee7128e59e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:23.379835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:23.379835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:23.379835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:23.379835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:23.379835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:23.379835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:23.379835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:23.379835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:23.379835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:23.379835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:23.379835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:23.379835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:23.379835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:23.379835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:23.379835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:23.381393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:23.381393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:23.381393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:23.381393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:23.381393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:23.381393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:23.381393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:23.381393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:23.381393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:23.381393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:23.381393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:23.381393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:23.381393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:23.381393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:23.381393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:23.381393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:23.382291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:23.382291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:23.382291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:23.382291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:23.382291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:23.382291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:23.382291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:23.382291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:23.382291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:23.382291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:23.382291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:23.382291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:23.387996 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 314/619] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 9 22:49:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:23.393698 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc256bbe-33b6-4df6-9645-8cee7128e59e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:23.437218 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e966e3d-b8bc-4803-a846-b9faa359d340 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:23.523396 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc256bbe-33b6-4df6-9645-8cee7128e59e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:23.524754 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f3964-cc88-4a9f-b8eb-6b7c1d719656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:23.525647 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f3964-cc88-4a9f-b8eb-6b7c1d719656 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:23.526028 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f3964-cc88-4a9f-b8eb-6b7c1d719656 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:23.526428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f3964-cc88-4a9f-b8eb-6b7c1d719656 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:23.537002 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f3964-cc88-4a9f-b8eb-6b7c1d719656 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:23.538875 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f3964-cc88-4a9f-b8eb-6b7c1d719656 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:23.557283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-a50f3964-cc88-4a9f-b8eb-6b7c1d719656 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:23.557283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:23.557283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:23.557283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:23.557283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:23.557283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:23.557283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:23.557283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:23.557283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:23.557283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:23.557283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:23.557283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:23.557283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:23.557283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:23.557283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:23.558412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:23.558412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:23.558412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:23.558412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:23.558412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:23.558412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:23.558412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:23.558412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:23.558412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:23.558412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:23.558412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:23.558412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:23.558412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:23.558412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:23.558412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:23.558412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:23.558412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:23.559458 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:23.559458 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:23.559458 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:23.559458 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:23.561079 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 306/620] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:23.569313 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a52695-d48f-46d7-9cee-b109ceeeba4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:23.569994 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a52695-d48f-46d7-9cee-b109ceeeba4c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:23.570296 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a52695-d48f-46d7-9cee-b109ceeeba4c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:23.570566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a52695-d48f-46d7-9cee-b109ceeeba4c None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:23.571418 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f3964-cc88-4a9f-b8eb-6b7c1d719656 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:23.571544 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a52695-d48f-46d7-9cee-b109ceeeba4c None admin] RBAC: Adding query filter params (name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:23.571790 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a52695-d48f-46d7-9cee-b109ceeeba4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:23.573170 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a52695-d48f-46d7-9cee-b109ceeeba4c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:23.597191 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 315/621] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 9 22:49:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:23.603352 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a52695-d48f-46d7-9cee-b109ceeeba4c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:23.761614 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f3964-cc88-4a9f-b8eb-6b7c1d719656 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:23.777264 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a52695-d48f-46d7-9cee-b109ceeeba4c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:23.820785 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796807b-62c5-4774-b3df-9f1438090ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:23.821462 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796807b-62c5-4774-b3df-9f1438090ccc None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:23.821741 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796807b-62c5-4774-b3df-9f1438090ccc None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:23.822457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796807b-62c5-4774-b3df-9f1438090ccc None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:23.823591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796807b-62c5-4774-b3df-9f1438090ccc None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:23.823961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796807b-62c5-4774-b3df-9f1438090ccc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:23.825806 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796807b-62c5-4774-b3df-9f1438090ccc None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:23.839658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec17ae0-5110-4ce3-8e2b-7fd999c24edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:23.840068 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec17ae0-5110-4ce3-8e2b-7fd999c24edf None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:23.840349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec17ae0-5110-4ce3-8e2b-7fd999c24edf None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:23.840619 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec17ae0-5110-4ce3-8e2b-7fd999c24edf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:23.853239 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 307/622] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:23.854141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796807b-62c5-4774-b3df-9f1438090ccc None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:23.861308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec17ae0-5110-4ce3-8e2b-7fd999c24edf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:23.862818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec17ae0-5110-4ce3-8e2b-7fd999c24edf None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:23.893202 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-0ec17ae0-5110-4ce3-8e2b-7fd999c24edf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:23.893202 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:23.893202 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:23.893202 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:23.893202 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:23.893202 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:23.893202 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:23.893202 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:23.893202 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:23.893202 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:23.893202 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:23.893202 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:23.893202 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:23.893202 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:23.893202 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:23.894199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:23.894199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:23.894199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:23.894199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:23.894199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:23.894199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:23.894199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:23.894199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:23.894199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:23.894199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:23.894199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:23.894199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:23.894199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:23.894199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:23.894199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:23.894199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:23.894199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:23.895140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:23.895140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:23.895140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:23.895140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:23.895140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 316/623] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:23.899625 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf28ee-b178-4609-b5e7-bf1174d09630 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:23.899625 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf28ee-b178-4609-b5e7-bf1174d09630 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:23.899625 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf28ee-b178-4609-b5e7-bf1174d09630 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:23.899625 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 317/624] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:23.903287 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf28ee-b178-4609-b5e7-bf1174d09630 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:23.979929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796807b-62c5-4774-b3df-9f1438090ccc None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:24.012968 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf28ee-b178-4609-b5e7-bf1174d09630 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:24.016447 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa8f2b-6edf-444a-b4ba-e2503ab41c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:24.017496 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa8f2b-6edf-444a-b4ba-e2503ab41c50 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:24.017984 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa8f2b-6edf-444a-b4ba-e2503ab41c50 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:24.018487 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa8f2b-6edf-444a-b4ba-e2503ab41c50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:24.025693 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa8f2b-6edf-444a-b4ba-e2503ab41c50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:24.027478 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa8f2b-6edf-444a-b4ba-e2503ab41c50 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:24.042427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-89aa8f2b-6edf-444a-b4ba-e2503ab41c50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:24.042427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:24.042427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:24.042427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:24.042427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:24.042427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:24.042427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:24.042427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:24.042427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:24.042427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:24.042427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:24.042427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:24.042427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:24.042427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:24.042427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:24.043355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:24.043355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:24.043355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:24.043355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:24.043355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:24.043355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:24.043355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:24.043355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:24.043355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:24.043355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:24.043355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:24.043355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:24.043355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:24.043355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:24.043355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:24.043355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:24.043355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:24.044251 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:24.044251 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:24.044251 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:24.044251 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:24.044251 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:24.044251 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:24.044251 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:24.044251 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:24.046411 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 308/625] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:24.047730 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96becb68-806b-40cc-b4c2-02348e3b4cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:24.048574 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96becb68-806b-40cc-b4c2-02348e3b4cea None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:24.049038 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96becb68-806b-40cc-b4c2-02348e3b4cea None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:24.049480 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96becb68-806b-40cc-b4c2-02348e3b4cea None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:24.050381 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96becb68-806b-40cc-b4c2-02348e3b4cea None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:24.050861 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96becb68-806b-40cc-b4c2-02348e3b4cea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:24.051488 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eff8d8-cef5-4c17-b1ce-d4c25d1f3d49 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:24.051850 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eff8d8-cef5-4c17-b1ce-d4c25d1f3d49 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:24.052832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eff8d8-cef5-4c17-b1ce-d4c25d1f3d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:24.053712 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96becb68-806b-40cc-b4c2-02348e3b4cea None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:24.069043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 318/626] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:24.070273 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96becb68-806b-40cc-b4c2-02348e3b4cea None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:24.071363 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-50eff8d8-cef5-4c17-b1ce-d4c25d1f3d49 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:24.072614 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-50eff8d8-cef5-4c17-b1ce-d4c25d1f3d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:24.150137 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 309/627] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:49:24.150137 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eff8d8-cef5-4c17-b1ce-d4c25d1f3d49 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:24.198766 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96becb68-806b-40cc-b4c2-02348e3b4cea None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:24.238277 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2491b-bb95-4cd0-9f6c-02990fdcd639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:24.238997 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2491b-bb95-4cd0-9f6c-02990fdcd639 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:24.239283 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2491b-bb95-4cd0-9f6c-02990fdcd639 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:24.240101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2491b-bb95-4cd0-9f6c-02990fdcd639 None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:24.240886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd2491b-bb95-4cd0-9f6c-02990fdcd639 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:24.241202 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd2491b-bb95-4cd0-9f6c-02990fdcd639 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:24.244966 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd2491b-bb95-4cd0-9f6c-02990fdcd639 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:24.257491 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 319/628] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:24.263100 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2491b-bb95-4cd0-9f6c-02990fdcd639 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:24.264429 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eff8d8-cef5-4c17-b1ce-d4c25d1f3d49 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:24.299887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f43a9b-7367-49ff-9cc0-c9b4a3ed4e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:24.301030 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f43a9b-7367-49ff-9cc0-c9b4a3ed4e82 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:24.301511 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f43a9b-7367-49ff-9cc0-c9b4a3ed4e82 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:24.302040 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f43a9b-7367-49ff-9cc0-c9b4a3ed4e82 None admin] PATH_INFO: `/v3/users/swift` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:24.310083 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f43a9b-7367-49ff-9cc0-c9b4a3ed4e82 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:24.311813 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f43a9b-7367-49ff-9cc0-c9b4a3ed4e82 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:24.326573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-f2f43a9b-7367-49ff-9cc0-c9b4a3ed4e82 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 22:49:24.326573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:24.326573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:24.326573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:24.326573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:24.326573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:24.326573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:24.326573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:24.326573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:24.326573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:24.326573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:24.326573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:24.326573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:24.326573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:24.326573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:24.326573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:24.327534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:24.327534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:24.327534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:24.327534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:24.327534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:24.327534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:24.327534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:24.327534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:24.327534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:24.327534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:24.327534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:24.327534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:24.327534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:24.327534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:24.327534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:24.327534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:24.327534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:24.327534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:24.328490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:24.328490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:24.328490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:24.328490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:24.328490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:24.328490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:24.328490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:24.328490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:24.328490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 22:49:24.328490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:24.331147 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 310/629] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:49:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:24.332211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f43a9b-7367-49ff-9cc0-c9b4a3ed4e82 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:24.367650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2491b-bb95-4cd0-9f6c-02990fdcd639 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:24.406355 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1adfe-e6ab-4a78-9ebe-c022428d5ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:24.407853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1adfe-e6ab-4a78-9ebe-c022428d5ac9 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:24.408567 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1adfe-e6ab-4a78-9ebe-c022428d5ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:24.409050 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1adfe-e6ab-4a78-9ebe-c022428d5ac9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:24.423465 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1adfe-e6ab-4a78-9ebe-c022428d5ac9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:24.434385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-88a1adfe-e6ab-4a78-9ebe-c022428d5ac9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:24.434385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:24.434385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:24.434385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:24.434385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:24.434385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:24.434385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:24.434385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:24.434385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:24.434385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:24.434385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:24.434385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:24.434385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:24.434385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:24.434385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:24.434385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:24.435323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:24.435323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:24.435323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:24.435323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:24.435323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:24.435323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:24.435323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:24.435323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:24.435323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:24.435323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:24.435323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:24.435323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:24.435323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:24.435323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:24.435323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:24.435323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:24.436332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:24.436332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:24.436332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:24.436332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:24.436332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:24.436332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:24.436332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:24.436332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:24.436332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:24.436332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:24.436332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:24.436332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:24.443922 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 320/630] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 9 22:49:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:24.443922 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1adfe-e6ab-4a78-9ebe-c022428d5ac9 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:24.456587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f43a9b-7367-49ff-9cc0-c9b4a3ed4e82 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:24.506563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08c8e6-f811-4546-bcbf-d98723770f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:24.507868 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08c8e6-f811-4546-bcbf-d98723770f57 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:24.508689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08c8e6-f811-4546-bcbf-d98723770f57 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:24.509448 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08c8e6-f811-4546-bcbf-d98723770f57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:24.510691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08c8e6-f811-4546-bcbf-d98723770f57 None admin] RBAC: Adding query filter params (role.id=44847d50e6cb4bc7938c59b4e2d99833, scope.project.id=0500e378ea374ca692e230bbd8781318, user.id=02fcd7cc86134ce488c7dd75dfb174b8) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:24.511506 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08c8e6-f811-4546-bcbf-d98723770f57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:24.513568 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08c8e6-f811-4546-bcbf-d98723770f57 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:24.522778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 311/631] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:49:24 2022] GET /identity/v3/role_assignments?user.id=02fcd7cc86134ce488c7dd75dfb174b8&scope.project.id=0500e378ea374ca692e230bbd8781318&role.id=44847d50e6cb4bc7938c59b4e2d99833 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:24.523878 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08c8e6-f811-4546-bcbf-d98723770f57 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:24.617327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1adfe-e6ab-4a78-9ebe-c022428d5ac9 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:24.639739 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08c8e6-f811-4546-bcbf-d98723770f57 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:24.680904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b540d-fc77-4c78-b0a6-b4cb3c103ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:24.681594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b540d-fc77-4c78-b0a6-b4cb3c103ff2 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:24.681872 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b540d-fc77-4c78-b0a6-b4cb3c103ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:24.682169 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b540d-fc77-4c78-b0a6-b4cb3c103ff2 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:24.682924 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58b540d-fc77-4c78-b0a6-b4cb3c103ff2 None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:24.683226 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58b540d-fc77-4c78-b0a6-b4cb3c103ff2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:24.684837 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58b540d-fc77-4c78-b0a6-b4cb3c103ff2 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:24.696187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 312/632] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 9 22:49:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:24.702082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b540d-fc77-4c78-b0a6-b4cb3c103ff2 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:24.714992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6c235-64ff-4d36-be6a-6f8242beab8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:24.716076 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6c235-64ff-4d36-be6a-6f8242beab8f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:24.716573 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6c235-64ff-4d36-be6a-6f8242beab8f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:24.717073 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6c235-64ff-4d36-be6a-6f8242beab8f None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:24.718144 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6c235-64ff-4d36-be6a-6f8242beab8f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:24.718698 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6c235-64ff-4d36-be6a-6f8242beab8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:24.720343 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6c235-64ff-4d36-be6a-6f8242beab8f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:24.756714 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 321/633] 10.222.0.23 () {56 vars in 1135 bytes} [Mon May 9 22:49:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:24.762319 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6c235-64ff-4d36-be6a-6f8242beab8f None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:24.822820 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b540d-fc77-4c78-b0a6-b4cb3c103ff2 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:24.868438 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576c14e-aabf-4ea5-8b67-6e1142573364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:24.869186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576c14e-aabf-4ea5-8b67-6e1142573364 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:24.869435 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576c14e-aabf-4ea5-8b67-6e1142573364 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:24.869708 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576c14e-aabf-4ea5-8b67-6e1142573364 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:24.876455 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6c235-64ff-4d36-be6a-6f8242beab8f None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:24.882594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576c14e-aabf-4ea5-8b67-6e1142573364 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:24.885128 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576c14e-aabf-4ea5-8b67-6e1142573364 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:24.906107 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-2576c14e-aabf-4ea5-8b67-6e1142573364 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:24.906107 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:24.906107 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:24.906107 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:24.906107 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:24.906107 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:24.906107 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:24.906107 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:24.906107 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:24.906107 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:24.906107 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:24.906107 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:24.906107 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:24.906107 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:24.906107 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:24.907186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:24.907186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:24.907186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:24.907186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:24.907186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:24.907186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:24.907186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:24.907186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:24.907186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:24.907186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:24.907186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:24.907186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:24.907186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:24.907186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:24.907186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:24.907186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:24.907186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:24.908218 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:24.908218 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:24.908218 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:24.908218 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:24.908218 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 313/634] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:24.915257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576c14e-aabf-4ea5-8b67-6e1142573364 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:24.919429 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df668ff4-8a65-4554-bbb9-c8017c489e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:24.920658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df668ff4-8a65-4554-bbb9-c8017c489e2f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:24.921239 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df668ff4-8a65-4554-bbb9-c8017c489e2f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:24.921726 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df668ff4-8a65-4554-bbb9-c8017c489e2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:24.934418 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df668ff4-8a65-4554-bbb9-c8017c489e2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:24.936260 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df668ff4-8a65-4554-bbb9-c8017c489e2f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:24.955157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-df668ff4-8a65-4554-bbb9-c8017c489e2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:24.955157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:24.955157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:24.955157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:24.955157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:24.955157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:24.955157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:24.955157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:24.955157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:24.955157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:24.955157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:24.955157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:24.955157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:24.955157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:24.955157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:24.956139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:24.956139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:24.956139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:24.956139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:24.956139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:24.956139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:24.956139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:24.956139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:24.956139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:24.956139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:24.956139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:24.956139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:24.956139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:24.956139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:24.956139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:24.956139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:24.956139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:24.957029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:24.957029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:24.957029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:24.957029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:24.959353 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 322/635] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:24.965044 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df668ff4-8a65-4554-bbb9-c8017c489e2f None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:25.018655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576c14e-aabf-4ea5-8b67-6e1142573364 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:25.059099 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d7c14-dd75-4eac-87d6-0d8df294dcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:25.059830 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d7c14-dd75-4eac-87d6-0d8df294dcba None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:25.060110 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d7c14-dd75-4eac-87d6-0d8df294dcba None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:25.060718 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d7c14-dd75-4eac-87d6-0d8df294dcba None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:25.061093 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d7c14-dd75-4eac-87d6-0d8df294dcba None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:25.061399 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d7c14-dd75-4eac-87d6-0d8df294dcba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:25.062811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d7c14-dd75-4eac-87d6-0d8df294dcba None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:25.075185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 314/636] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:25.076341 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d12be-205b-4bd5-9ff7-64463eb684b9 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:25.076627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d12be-205b-4bd5-9ff7-64463eb684b9 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:25.076898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d12be-205b-4bd5-9ff7-64463eb684b9 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:25.078252 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 315/637] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:25.081838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d12be-205b-4bd5-9ff7-64463eb684b9 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:25.084437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df668ff4-8a65-4554-bbb9-c8017c489e2f None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:25.120153 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a18be-dbcb-4a1f-9bf5-3dd79c9de0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:25.120735 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a18be-dbcb-4a1f-9bf5-3dd79c9de0a3 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:25.121013 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a18be-dbcb-4a1f-9bf5-3dd79c9de0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:25.121281 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a18be-dbcb-4a1f-9bf5-3dd79c9de0a3 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:25.121995 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a18be-dbcb-4a1f-9bf5-3dd79c9de0a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:25.122362 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a18be-dbcb-4a1f-9bf5-3dd79c9de0a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:25.123702 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a18be-dbcb-4a1f-9bf5-3dd79c9de0a3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:25.135260 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 323/638] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:25.136496 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e317a-f7de-415b-a6a1-4b67d2185c3f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:25.136782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e317a-f7de-415b-a6a1-4b67d2185c3f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:25.137057 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e317a-f7de-415b-a6a1-4b67d2185c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:25.152835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b80e317a-f7de-415b-a6a1-4b67d2185c3f None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:25.154018 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b80e317a-f7de-415b-a6a1-4b67d2185c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:25.189371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d12be-205b-4bd5-9ff7-64463eb684b9 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:25.220311 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 324/639] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:49:25.221623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabbefc-a978-4b0c-a330-1ecd50d04a08 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:25.221909 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabbefc-a978-4b0c-a330-1ecd50d04a08 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:25.226227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabbefc-a978-4b0c-a330-1ecd50d04a08 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:25.226376 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d611e-27e7-4b44-858a-ee8702057ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:25.226376 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d611e-27e7-4b44-858a-ee8702057ea8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:25.226550 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 325/640] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:25.226550 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dabbefc-a978-4b0c-a330-1ecd50d04a08 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:25.228042 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d611e-27e7-4b44-858a-ee8702057ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:25.228350 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d611e-27e7-4b44-858a-ee8702057ea8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:25.236310 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d611e-27e7-4b44-858a-ee8702057ea8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:25.237804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d611e-27e7-4b44-858a-ee8702057ea8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:25.252668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-768d611e-27e7-4b44-858a-ee8702057ea8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 22:49:25.252668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:25.252668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:25.252668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:25.252668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:25.252668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:25.252668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:25.252668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:25.252668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:25.252668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:25.252668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:25.252668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:25.252668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:25.252668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:25.252668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:25.252668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:25.253698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:25.253698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:25.253698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:25.253698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:25.253698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:25.253698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:25.253698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:25.253698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:25.253698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:25.253698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:25.253698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:25.253698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:25.253698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:25.253698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:25.253698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:25.253698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:25.253698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:25.253698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:25.254714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:25.254714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:25.254714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:25.254714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:25.254714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:25.254714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:25.254714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:25.254714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:25.254714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 22:49:25.254714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:25.254714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 316/641] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:49:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:25.255378 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834eec2-695e-4016-9c86-154774b3c8e5 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:25.255454 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834eec2-695e-4016-9c86-154774b3c8e5 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:25.256039 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834eec2-695e-4016-9c86-154774b3c8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:25.273061 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-7834eec2-695e-4016-9c86-154774b3c8e5 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:25.274236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7834eec2-695e-4016-9c86-154774b3c8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:25.338911 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dabbefc-a978-4b0c-a330-1ecd50d04a08 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:25.341907 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 317/642] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:49:25.342690 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834eec2-695e-4016-9c86-154774b3c8e5 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:25.385078 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6dd3ac-cf15-4815-9cd5-d6fea487182e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:25.385790 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dd3ac-cf15-4815-9cd5-d6fea487182e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:25.386070 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dd3ac-cf15-4815-9cd5-d6fea487182e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:25.386366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dd3ac-cf15-4815-9cd5-d6fea487182e None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:25.392980 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6dd3ac-cf15-4815-9cd5-d6fea487182e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:25.394443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6dd3ac-cf15-4815-9cd5-d6fea487182e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:25.408892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-dd6dd3ac-cf15-4815-9cd5-d6fea487182e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:25.408892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:25.408892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:25.408892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:25.408892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:25.408892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:25.408892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:25.408892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:25.408892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:25.408892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:25.408892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:25.408892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:25.408892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:25.408892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:25.408892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:25.409871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:25.409871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:25.409871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:25.409871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:25.409871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:25.409871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:25.409871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:25.409871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:25.409871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:25.409871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:25.409871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:25.409871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:25.409871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:25.409871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:25.409871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:25.409871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:25.409871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:25.410942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:25.410942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:25.410942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:25.410942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:25.410942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:25.410942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:25.410942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:25.410942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:25.415194 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 326/643] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:25.416002 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6dd3ac-cf15-4815-9cd5-d6fea487182e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:25.444071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834eec2-695e-4016-9c86-154774b3c8e5 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:25.477921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee270f-8cec-4309-bd08-a75a67a539eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:25.478741 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee270f-8cec-4309-bd08-a75a67a539eb None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:25.479024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee270f-8cec-4309-bd08-a75a67a539eb None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:25.479299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee270f-8cec-4309-bd08-a75a67a539eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:25.489303 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ee270f-8cec-4309-bd08-a75a67a539eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:25.497561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-a2ee270f-8cec-4309-bd08-a75a67a539eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:25.497561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:25.497561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:25.497561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:25.497561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:25.497561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:25.497561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:25.497561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:25.497561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:25.497561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:25.497561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:25.497561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:25.497561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:25.497561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:25.497561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:25.497561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:25.498558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:25.498558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:25.498558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:25.498558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:25.498558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:25.498558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:25.498558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:25.498558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:25.498558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:25.498558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:25.498558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:25.498558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:25.498558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:25.498558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:25.498558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:25.498558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:25.499473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:25.499473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:25.499473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:25.499473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:25.499473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:25.499473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:25.499473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:25.499473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:25.499473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:25.499473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:25.499473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:25.499473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:25.499473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 318/644] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 9 22:49:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:25.500801 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee270f-8cec-4309-bd08-a75a67a539eb None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:25.527234 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6dd3ac-cf15-4815-9cd5-d6fea487182e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:25.560345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e81e16-7a7e-4f4f-b94c-96a31c5d0dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:25.561016 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e81e16-7a7e-4f4f-b94c-96a31c5d0dc3 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:25.561458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e81e16-7a7e-4f4f-b94c-96a31c5d0dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:25.561744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e81e16-7a7e-4f4f-b94c-96a31c5d0dc3 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:25.563074 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e81e16-7a7e-4f4f-b94c-96a31c5d0dc3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:25.563387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e81e16-7a7e-4f4f-b94c-96a31c5d0dc3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:25.565241 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e81e16-7a7e-4f4f-b94c-96a31c5d0dc3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:25.583428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 327/645] 10.222.0.23 () {56 vars in 1137 bytes} [Mon May 9 22:49:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:25.584535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e81e16-7a7e-4f4f-b94c-96a31c5d0dc3 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:25.599981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee270f-8cec-4309-bd08-a75a67a539eb None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:25.638045 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572b014-49e5-4e65-9e23-745eaf6f91e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:25.638795 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572b014-49e5-4e65-9e23-745eaf6f91e6 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:25.639087 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572b014-49e5-4e65-9e23-745eaf6f91e6 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:25.639400 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572b014-49e5-4e65-9e23-745eaf6f91e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:25.649662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3572b014-49e5-4e65-9e23-745eaf6f91e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:25.657320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-3572b014-49e5-4e65-9e23-745eaf6f91e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:25.657320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:25.657320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:25.657320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:25.657320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:25.657320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:25.657320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:25.657320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:25.657320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:25.657320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:25.657320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:25.657320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:25.657320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:25.657320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:25.657320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:25.657320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:25.658332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:25.658332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:25.658332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:25.658332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:25.658332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:25.658332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:25.658332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:25.658332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:25.658332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:25.658332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:25.658332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:25.658332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:25.658332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:25.658332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:25.658332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:25.658332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:25.659201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:25.659201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:25.659201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:25.659201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:25.659201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:25.659201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:25.659201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:25.659201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:25.659201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:25.659201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:25.659201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:25.659201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:25.659201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 319/646] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 9 22:49:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:25.660318 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572b014-49e5-4e65-9e23-745eaf6f91e6 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:25.694878 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e81e16-7a7e-4f4f-b94c-96a31c5d0dc3 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:25.727953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91243005-39c1-453f-b557-0e2c75a5f3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:25.728619 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91243005-39c1-453f-b557-0e2c75a5f3b4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:25.728901 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91243005-39c1-453f-b557-0e2c75a5f3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:25.729180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91243005-39c1-453f-b557-0e2c75a5f3b4 None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:25.729966 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91243005-39c1-453f-b557-0e2c75a5f3b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:25.730315 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91243005-39c1-453f-b557-0e2c75a5f3b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:25.731732 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91243005-39c1-453f-b557-0e2c75a5f3b4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:25.743850 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 328/647] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:25.744908 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91243005-39c1-453f-b557-0e2c75a5f3b4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:25.767161 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572b014-49e5-4e65-9e23-745eaf6f91e6 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:25.799758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb351bc-dee7-4f29-99d3-1db11df89a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:25.800444 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb351bc-dee7-4f29-99d3-1db11df89a44 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:25.800722 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb351bc-dee7-4f29-99d3-1db11df89a44 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:25.800989 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb351bc-dee7-4f29-99d3-1db11df89a44 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:25.801715 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb351bc-dee7-4f29-99d3-1db11df89a44 None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:25.802601 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb351bc-dee7-4f29-99d3-1db11df89a44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:25.803946 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb351bc-dee7-4f29-99d3-1db11df89a44 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:25.815871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 320/648] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 9 22:49:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:25.815871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb351bc-dee7-4f29-99d3-1db11df89a44 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:25.839421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91243005-39c1-453f-b557-0e2c75a5f3b4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:25.871374 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e768ef-add1-4a68-b6e5-dec53c740456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:25.872005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e768ef-add1-4a68-b6e5-dec53c740456 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:25.872280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e768ef-add1-4a68-b6e5-dec53c740456 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:25.872553 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e768ef-add1-4a68-b6e5-dec53c740456 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:25.914921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb351bc-dee7-4f29-99d3-1db11df89a44 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:25.940625 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e768ef-add1-4a68-b6e5-dec53c740456 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:25.942865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e768ef-add1-4a68-b6e5-dec53c740456 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:25.947673 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a5536-0622-4429-9370-7c503fd2efd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:25.948329 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a5536-0622-4429-9370-7c503fd2efd9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:25.948605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a5536-0622-4429-9370-7c503fd2efd9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:25.948873 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a5536-0622-4429-9370-7c503fd2efd9 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:25.949609 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66a5536-0622-4429-9370-7c503fd2efd9 None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:25.949911 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66a5536-0622-4429-9370-7c503fd2efd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:25.951288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66a5536-0622-4429-9370-7c503fd2efd9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:25.963974 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 321/649] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 9 22:49:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:25.964792 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a5536-0622-4429-9370-7c503fd2efd9 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:26.038860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 329/650] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:49:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:49:26.039747 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e768ef-add1-4a68-b6e5-dec53c740456 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:26.066604 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a5536-0622-4429-9370-7c503fd2efd9 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:26.099528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c2e98-eafa-42fb-8f43-5e6385fb0b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:26.100688 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c2e98-eafa-42fb-8f43-5e6385fb0b1c None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:26.100966 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c2e98-eafa-42fb-8f43-5e6385fb0b1c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:26.101241 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c2e98-eafa-42fb-8f43-5e6385fb0b1c None admin] PATH_INFO: `/v3/projects/0500e378ea374ca692e230bbd8781318/users/8cb6c294ce1842b68a6bc73ac381ed6b/roles/44847d50e6cb4bc7938c59b4e2d99833` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:26.116239 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c2e98-eafa-42fb-8f43-5e6385fb0b1c None admin] RBAC: Authorizing `identity:create_grant(project_id=0500e378ea374ca692e230bbd8781318, user_id=8cb6c294ce1842b68a6bc73ac381ed6b, role_id=44847d50e6cb4bc7938c59b4e2d99833)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:26.117672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c2e98-eafa-42fb-8f43-5e6385fb0b1c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:26.142145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 322/651] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:49:25 2022] PUT /identity/v3/projects/0500e378ea374ca692e230bbd8781318/users/8cb6c294ce1842b68a6bc73ac381ed6b/roles/44847d50e6cb4bc7938c59b4e2d99833 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:49:26.142145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c2e98-eafa-42fb-8f43-5e6385fb0b1c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:26.157580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e768ef-add1-4a68-b6e5-dec53c740456 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:26.200322 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d9432-df0a-473f-9257-b8052c69729a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:26.201043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d9432-df0a-473f-9257-b8052c69729a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:26.201320 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d9432-df0a-473f-9257-b8052c69729a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:26.201593 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d9432-df0a-473f-9257-b8052c69729a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:26.213105 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d9432-df0a-473f-9257-b8052c69729a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:26.214626 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d9432-df0a-473f-9257-b8052c69729a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:26.233511 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-484d9432-df0a-473f-9257-b8052c69729a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:26.233511 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:26.233511 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:26.233511 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:26.233511 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:26.233511 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:26.233511 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:26.233511 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:26.233511 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:26.233511 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:26.233511 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:26.233511 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:26.233511 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:26.233511 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:26.233511 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:26.234816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:26.234816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:26.234816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:26.234816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:26.234816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:26.234816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:26.234816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:26.234816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:26.234816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:26.234816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:26.234816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:26.234816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:26.234816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:26.234816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:26.234816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:26.234816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:26.234816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:26.236164 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:26.236164 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:26.236164 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:26.236164 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:26.236164 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 330/652] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:26.236464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d9432-df0a-473f-9257-b8052c69729a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:26.239070 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c2e98-eafa-42fb-8f43-5e6385fb0b1c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:26.278035 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b105ea8-71b9-4412-beda-ddc64ad2d4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:26.278880 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b105ea8-71b9-4412-beda-ddc64ad2d4d1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:26.279155 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b105ea8-71b9-4412-beda-ddc64ad2d4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:26.279421 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b105ea8-71b9-4412-beda-ddc64ad2d4d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:26.294228 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b105ea8-71b9-4412-beda-ddc64ad2d4d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:26.295690 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b105ea8-71b9-4412-beda-ddc64ad2d4d1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:26.314520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-0b105ea8-71b9-4412-beda-ddc64ad2d4d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:26.314520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:26.314520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:26.314520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:26.314520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:26.314520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:26.314520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:26.314520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:26.314520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:26.314520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:26.314520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:26.314520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:26.314520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:26.314520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:26.314520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:26.315521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:26.315521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:26.315521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:26.315521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:26.315521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:26.315521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:26.315521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:26.315521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:26.315521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:26.315521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:26.315521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:26.315521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:26.315521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:26.315521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:26.315521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:26.315521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:26.315521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:26.316459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:26.316459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:26.316459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:26.316459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:26.321461 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 323/653] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:26.321976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b105ea8-71b9-4412-beda-ddc64ad2d4d1 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:26.351932 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d9432-df0a-473f-9257-b8052c69729a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:26.388261 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb767e1-121c-4854-97d3-c2057f952b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:26.389053 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb767e1-121c-4854-97d3-c2057f952b3d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:26.389427 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb767e1-121c-4854-97d3-c2057f952b3d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:26.389860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb767e1-121c-4854-97d3-c2057f952b3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:26.403432 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb767e1-121c-4854-97d3-c2057f952b3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:26.405317 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb767e1-121c-4854-97d3-c2057f952b3d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:26.425318 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b105ea8-71b9-4412-beda-ddc64ad2d4d1 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:26.428972 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-4bb767e1-121c-4854-97d3-c2057f952b3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:26.428972 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:26.428972 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:26.428972 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:26.428972 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:26.428972 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:26.428972 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:26.428972 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:26.428972 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:26.428972 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:26.428972 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:26.428972 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:26.428972 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:26.428972 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:26.428972 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:26.429867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:26.429867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:26.429867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:26.429867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:26.429867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:26.429867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:26.429867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:26.429867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:26.429867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:26.429867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:26.429867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:26.429867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:26.429867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:26.429867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:26.429867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:26.429867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:26.429867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:26.430814 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:26.430814 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:26.430814 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:26.430814 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:26.434206 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 331/654] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:26.435050 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb767e1-121c-4854-97d3-c2057f952b3d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:26.462541 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b025e-b91f-4755-98e3-ec004134acb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:26.463169 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b025e-b91f-4755-98e3-ec004134acb8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:26.463500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b025e-b91f-4755-98e3-ec004134acb8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:26.463772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b025e-b91f-4755-98e3-ec004134acb8 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:26.464489 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b025e-b91f-4755-98e3-ec004134acb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:26.464787 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b025e-b91f-4755-98e3-ec004134acb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:26.466231 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b025e-b91f-4755-98e3-ec004134acb8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:26.479102 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 324/655] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:26.479937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b025e-b91f-4755-98e3-ec004134acb8 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:26.548639 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb767e1-121c-4854-97d3-c2057f952b3d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:26.582406 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b025e-b91f-4755-98e3-ec004134acb8 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:26.583527 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10365226-3339-4c9c-b245-030120c7756d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:26.584368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10365226-3339-4c9c-b245-030120c7756d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:26.584654 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10365226-3339-4c9c-b245-030120c7756d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:26.585748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10365226-3339-4c9c-b245-030120c7756d None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:26.586514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10365226-3339-4c9c-b245-030120c7756d None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:26.586824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10365226-3339-4c9c-b245-030120c7756d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:26.588208 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10365226-3339-4c9c-b245-030120c7756d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:26.604028 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 332/656] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:26.604899 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10365226-3339-4c9c-b245-030120c7756d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:26.637761 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d117d-a6d6-432b-9665-eeb2b20a0390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:26.638491 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d117d-a6d6-432b-9665-eeb2b20a0390 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:26.638810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d117d-a6d6-432b-9665-eeb2b20a0390 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:26.639077 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d117d-a6d6-432b-9665-eeb2b20a0390 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:26.639795 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d117d-a6d6-432b-9665-eeb2b20a0390 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:26.640118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d117d-a6d6-432b-9665-eeb2b20a0390 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:26.642302 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d117d-a6d6-432b-9665-eeb2b20a0390 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:26.664562 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 325/657] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:26.664562 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d117d-a6d6-432b-9665-eeb2b20a0390 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:26.726323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10365226-3339-4c9c-b245-030120c7756d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:26.767247 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b0654-9093-400a-971e-b06169db3a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:26.768561 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b0654-9093-400a-971e-b06169db3a95 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:26.769341 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b0654-9093-400a-971e-b06169db3a95 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:26.769824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b0654-9093-400a-971e-b06169db3a95 None admin] PATH_INFO: `/v3/users/glance` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:26.781239 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b0654-9093-400a-971e-b06169db3a95 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:26.783252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b0654-9093-400a-971e-b06169db3a95 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:26.783708 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d117d-a6d6-432b-9665-eeb2b20a0390 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:26.802366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-499b0654-9093-400a-971e-b06169db3a95 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 09 22:49:26.802366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:26.802366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:26.802366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:26.802366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:26.802366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:26.802366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:26.802366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:26.802366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:26.802366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:26.802366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:26.802366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:26.802366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:26.802366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:26.802366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:26.802366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:26.803349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:26.803349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:26.803349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:26.803349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:26.803349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:26.803349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:26.803349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:26.803349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:26.803349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:26.803349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:26.803349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:26.803349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:26.803349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:26.803349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:26.803349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:26.803349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:26.803349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:26.803349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:26.804299 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:26.804299 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:26.804299 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:26.804299 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:26.804299 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:26.804299 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:26.804299 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:26.804299 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:26.804299 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 09 22:49:26.804299 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:26.807057 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 333/658] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:49:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:26.811668 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b0654-9093-400a-971e-b06169db3a95 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:26.823475 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781ccdb-a58a-436b-8894-2b93a2af46f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:26.824312 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781ccdb-a58a-436b-8894-2b93a2af46f4 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:26.824588 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781ccdb-a58a-436b-8894-2b93a2af46f4 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:26.824859 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781ccdb-a58a-436b-8894-2b93a2af46f4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:26.832116 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3781ccdb-a58a-436b-8894-2b93a2af46f4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:26.834155 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3781ccdb-a58a-436b-8894-2b93a2af46f4 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:26.849439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-3781ccdb-a58a-436b-8894-2b93a2af46f4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 09 22:49:26.849439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:26.849439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:26.849439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:26.849439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:26.849439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:26.849439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:26.849439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:26.849439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:26.849439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:26.849439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:26.849439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:26.849439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:26.849439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:26.849439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:26.849439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:26.850469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:26.850469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:26.850469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:26.850469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:26.850469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:26.850469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:26.850469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:26.850469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:26.850469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:26.850469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:26.850469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:26.850469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:26.850469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:26.850469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:26.850469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:26.850469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:26.850469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:26.850469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:26.852024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:26.852024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:26.852024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:26.852024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:26.852024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:26.852024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:26.852024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:26.852024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:26.852024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 09 22:49:26.852024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:26.855295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 326/659] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 9 22:49:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:26.856251 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781ccdb-a58a-436b-8894-2b93a2af46f4 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:26.929754 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b0654-9093-400a-971e-b06169db3a95 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:26.961121 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781ccdb-a58a-436b-8894-2b93a2af46f4 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:26.964750 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527d9c0-495a-426b-be19-8589311e1d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:26.966417 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527d9c0-495a-426b-be19-8589311e1d33 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:26.966417 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527d9c0-495a-426b-be19-8589311e1d33 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:26.966417 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527d9c0-495a-426b-be19-8589311e1d33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:26.974578 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527d9c0-495a-426b-be19-8589311e1d33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:26.976108 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527d9c0-495a-426b-be19-8589311e1d33 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:26.991819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-5527d9c0-495a-426b-be19-8589311e1d33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:26.991819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:26.991819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:26.991819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:26.991819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:26.991819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:26.991819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:26.991819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:26.991819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:26.991819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:26.991819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:26.991819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:26.991819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:26.991819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:26.991819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:26.992772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:26.992772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:26.992772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:26.992772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:26.992772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:26.992772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:26.992772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:26.992772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:26.992772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:26.992772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:26.992772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:26.992772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:26.992772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:26.992772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:26.992772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:26.992772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:26.992772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:26.993682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:26.993682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:26.993682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:26.993682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:26.993682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:26.993682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:26.993682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:26.993682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:26.997634 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 334/660] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:26.998517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527d9c0-495a-426b-be19-8589311e1d33 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:27.005612 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0060a-cf87-450f-b18a-b5a582df5eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:27.006617 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0060a-cf87-450f-b18a-b5a582df5eea None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:27.007126 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0060a-cf87-450f-b18a-b5a582df5eea None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:27.007601 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0060a-cf87-450f-b18a-b5a582df5eea None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:27.008635 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c0060a-cf87-450f-b18a-b5a582df5eea None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:27.009161 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c0060a-cf87-450f-b18a-b5a582df5eea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:27.010884 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c0060a-cf87-450f-b18a-b5a582df5eea None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:27.033679 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 327/661] 10.222.0.23 () {56 vars in 1137 bytes} [Mon May 9 22:49:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:27.034921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0060a-cf87-450f-b18a-b5a582df5eea None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:27.098477 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527d9c0-495a-426b-be19-8589311e1d33 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:27.132489 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648839f8-6d6c-4ed6-bd0c-c862e80965e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:27.133130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648839f8-6d6c-4ed6-bd0c-c862e80965e2 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:27.133402 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648839f8-6d6c-4ed6-bd0c-c862e80965e2 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:27.133708 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648839f8-6d6c-4ed6-bd0c-c862e80965e2 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:27.134581 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648839f8-6d6c-4ed6-bd0c-c862e80965e2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:27.134882 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648839f8-6d6c-4ed6-bd0c-c862e80965e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:27.136289 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648839f8-6d6c-4ed6-bd0c-c862e80965e2 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:27.136829 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0060a-cf87-450f-b18a-b5a582df5eea None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:27.153722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 335/662] 10.222.0.23 () {56 vars in 1139 bytes} [Mon May 9 22:49:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:27.157076 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648839f8-6d6c-4ed6-bd0c-c862e80965e2 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:27.172023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c235b2-839d-44c2-9bb0-858900e8a907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:27.172680 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c235b2-839d-44c2-9bb0-858900e8a907 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:27.172958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c235b2-839d-44c2-9bb0-858900e8a907 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:27.173229 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c235b2-839d-44c2-9bb0-858900e8a907 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:27.173998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c235b2-839d-44c2-9bb0-858900e8a907 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:27.174328 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c235b2-839d-44c2-9bb0-858900e8a907 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:27.175732 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c235b2-839d-44c2-9bb0-858900e8a907 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:27.186750 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 328/663] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:27.187945 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293d2ff-07a9-4515-aae0-0824d6719db7 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:27.188224 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293d2ff-07a9-4515-aae0-0824d6719db7 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:27.188496 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293d2ff-07a9-4515-aae0-0824d6719db7 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:27.194412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 329/664] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:27.194412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293d2ff-07a9-4515-aae0-0824d6719db7 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:27.262976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648839f8-6d6c-4ed6-bd0c-c862e80965e2 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:27.290618 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293d2ff-07a9-4515-aae0-0824d6719db7 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:27.299616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5ffb8-8590-40ef-b654-ebbf238bbda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:27.300318 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5ffb8-8590-40ef-b654-ebbf238bbda5 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:27.300561 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5ffb8-8590-40ef-b654-ebbf238bbda5 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:27.300833 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5ffb8-8590-40ef-b654-ebbf238bbda5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:27.324810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f89767-e4d4-4cff-8ec1-f1ad85646f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:27.325463 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f89767-e4d4-4cff-8ec1-f1ad85646f9d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:27.325752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f89767-e4d4-4cff-8ec1-f1ad85646f9d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:27.326012 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f89767-e4d4-4cff-8ec1-f1ad85646f9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:27.368711 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5ffb8-8590-40ef-b654-ebbf238bbda5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:27.370181 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5ffb8-8590-40ef-b654-ebbf238bbda5 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:27.391983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f89767-e4d4-4cff-8ec1-f1ad85646f9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:27.393434 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f89767-e4d4-4cff-8ec1-f1ad85646f9d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:27.465224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 336/665] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:49:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:49:27.466062 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5ffb8-8590-40ef-b654-ebbf238bbda5 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:27.499328 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 330/666] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:49:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:49:27.499328 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b6fa4-ffa0-4281-aee6-6c7e1c7bec0d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:27.499328 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b6fa4-ffa0-4281-aee6-6c7e1c7bec0d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:27.499328 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b6fa4-ffa0-4281-aee6-6c7e1c7bec0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:27.515025 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-0b0b6fa4-ffa0-4281-aee6-6c7e1c7bec0d None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:27.516141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0b0b6fa4-ffa0-4281-aee6-6c7e1c7bec0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:27.563240 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5ffb8-8590-40ef-b654-ebbf238bbda5 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:27.590135 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 331/667] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:49:27.590135 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b6fa4-ffa0-4281-aee6-6c7e1c7bec0d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:27.596548 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6930fb9b-7100-42c4-8e68-af3d49139b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:27.597283 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930fb9b-7100-42c4-8e68-af3d49139b2c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:27.597632 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930fb9b-7100-42c4-8e68-af3d49139b2c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:27.598004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930fb9b-7100-42c4-8e68-af3d49139b2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:27.598987 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6930fb9b-7100-42c4-8e68-af3d49139b2c None admin] RBAC: Adding query filter params (role.id=44847d50e6cb4bc7938c59b4e2d99833, scope.project.id=0500e378ea374ca692e230bbd8781318, user.id=5fe668a2cee040c7a56d1f8685744cd1) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:27.599358 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6930fb9b-7100-42c4-8e68-af3d49139b2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:27.600856 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6930fb9b-7100-42c4-8e68-af3d49139b2c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:27.607119 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 337/668] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:49:27 2022] GET /identity/v3/role_assignments?user.id=5fe668a2cee040c7a56d1f8685744cd1&scope.project.id=0500e378ea374ca692e230bbd8781318&role.id=44847d50e6cb4bc7938c59b4e2d99833 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:27.608018 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6930fb9b-7100-42c4-8e68-af3d49139b2c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:27.703992 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b6fa4-ffa0-4281-aee6-6c7e1c7bec0d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:27.718252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6930fb9b-7100-42c4-8e68-af3d49139b2c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:27.746905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5db2ba-9f06-4eb6-be08-b25f00b20e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:27.747709 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5db2ba-9f06-4eb6-be08-b25f00b20e38 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:27.747982 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5db2ba-9f06-4eb6-be08-b25f00b20e38 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:27.748252 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5db2ba-9f06-4eb6-be08-b25f00b20e38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:27.757369 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1af55-0874-4c2f-a866-2eb5a40f2e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:27.758195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1af55-0874-4c2f-a866-2eb5a40f2e85 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:27.758479 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1af55-0874-4c2f-a866-2eb5a40f2e85 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:27.758744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1af55-0874-4c2f-a866-2eb5a40f2e85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:27.760453 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5db2ba-9f06-4eb6-be08-b25f00b20e38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:27.761911 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5db2ba-9f06-4eb6-be08-b25f00b20e38 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:27.770342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1af55-0874-4c2f-a866-2eb5a40f2e85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:27.772188 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1af55-0874-4c2f-a866-2eb5a40f2e85 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:27.780360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-8d5db2ba-9f06-4eb6-be08-b25f00b20e38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:27.780360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:27.780360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:27.780360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:27.780360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:27.780360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:27.780360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:27.780360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:27.780360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:27.780360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:27.780360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:27.780360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:27.780360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:27.780360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:27.780360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:27.781335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:27.781335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:27.781335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:27.781335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:27.781335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:27.781335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:27.781335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:27.781335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:27.781335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:27.781335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:27.781335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:27.781335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:27.781335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:27.781335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:27.781335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:27.781335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:27.781335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:27.782290 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:27.782290 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:27.782290 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:27.782290 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:27.786211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 332/669] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:27.787126 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5db2ba-9f06-4eb6-be08-b25f00b20e38 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:27.791258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-f1d1af55-0874-4c2f-a866-2eb5a40f2e85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:27.791258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:27.791258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:27.791258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:27.791258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:27.791258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:27.791258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:27.791258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:27.791258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:27.791258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:27.791258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:27.791258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:27.791258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:27.791258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:27.791258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:27.793388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:27.793388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:27.793388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:27.793388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:27.793388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:27.793388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:27.793388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:27.793388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:27.793388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:27.793388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:27.793388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:27.793388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:27.793388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:27.793388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:27.793388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:27.793388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:27.793388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:27.795724 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:27.795724 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:27.795724 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:27.795724 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:27.795724 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 338/670] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:27.795724 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1af55-0874-4c2f-a866-2eb5a40f2e85 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:27.903297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5db2ba-9f06-4eb6-be08-b25f00b20e38 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:27.907863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1af55-0874-4c2f-a866-2eb5a40f2e85 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:27.941958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b83f9-f3f6-488d-b7e3-166f7e691fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:27.942809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b83f9-f3f6-488d-b7e3-166f7e691fae None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:27.943083 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b83f9-f3f6-488d-b7e3-166f7e691fae None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:27.943351 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b83f9-f3f6-488d-b7e3-166f7e691fae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:27.953609 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8281050f-5d94-43e1-a1e6-938ff89c44bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:27.953883 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b83f9-f3f6-488d-b7e3-166f7e691fae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:27.954301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8281050f-5d94-43e1-a1e6-938ff89c44bc None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:27.954581 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8281050f-5d94-43e1-a1e6-938ff89c44bc None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:27.954852 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8281050f-5d94-43e1-a1e6-938ff89c44bc None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:27.955573 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8281050f-5d94-43e1-a1e6-938ff89c44bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:27.955875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8281050f-5d94-43e1-a1e6-938ff89c44bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:27.957231 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8281050f-5d94-43e1-a1e6-938ff89c44bc None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:27.961617 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-a95b83f9-f3f6-488d-b7e3-166f7e691fae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:49:27.961617 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:27.961617 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:27.961617 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:27.961617 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:27.961617 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:27.961617 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:27.961617 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:27.961617 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:27.961617 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:27.961617 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:27.961617 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:27.961617 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:27.961617 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:27.961617 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:27.961617 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:27.962899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:27.962899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:27.962899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:27.962899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:27.962899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:27.962899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:27.962899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:27.962899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:27.962899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:27.962899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:27.962899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:27.962899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:27.962899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:27.962899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:27.962899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:27.962899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:27.964095 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:27.964095 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:27.964095 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:27.964095 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:27.964095 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:27.964095 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:27.964095 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:27.964095 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:27.964095 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:27.964095 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:27.964095 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:49:27.964095 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:27.964095 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 333/671] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:49:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:27.964095 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b83f9-f3f6-488d-b7e3-166f7e691fae None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:27.969051 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 339/672] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:27.969884 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8281050f-5d94-43e1-a1e6-938ff89c44bc None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:28.079194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b83f9-f3f6-488d-b7e3-166f7e691fae None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:28.101913 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8281050f-5d94-43e1-a1e6-938ff89c44bc None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:28.118454 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf790a-72bf-4337-82be-3f93bb8a8906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:28.119874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf790a-72bf-4337-82be-3f93bb8a8906 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:28.120384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf790a-72bf-4337-82be-3f93bb8a8906 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:28.120876 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf790a-72bf-4337-82be-3f93bb8a8906 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:28.121794 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf790a-72bf-4337-82be-3f93bb8a8906 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:28.122773 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf790a-72bf-4337-82be-3f93bb8a8906 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:28.124795 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf790a-72bf-4337-82be-3f93bb8a8906 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:28.135443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ba601-21dd-4ccd-abf7-c43bef28723f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:28.136105 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ba601-21dd-4ccd-abf7-c43bef28723f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:28.136378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ba601-21dd-4ccd-abf7-c43bef28723f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:28.136648 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ba601-21dd-4ccd-abf7-c43bef28723f None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:28.137542 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ba601-21dd-4ccd-abf7-c43bef28723f None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:28.137740 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ba601-21dd-4ccd-abf7-c43bef28723f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:28.139167 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ba601-21dd-4ccd-abf7-c43bef28723f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:28.141800 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 334/673] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:28.142920 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf790a-72bf-4337-82be-3f93bb8a8906 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:28.156473 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 340/674] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:49:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:28.157510 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ba601-21dd-4ccd-abf7-c43bef28723f None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:28.262531 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ba601-21dd-4ccd-abf7-c43bef28723f None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:28.263016 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf790a-72bf-4337-82be-3f93bb8a8906 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:28.296820 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265c9e0-af65-4525-8614-711277a5a1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:28.297660 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265c9e0-af65-4525-8614-711277a5a1f0 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:28.297934 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265c9e0-af65-4525-8614-711277a5a1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:28.298226 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265c9e0-af65-4525-8614-711277a5a1f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:28.299414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe6fda-50ac-43c6-9230-538279bbe7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:28.300425 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe6fda-50ac-43c6-9230-538279bbe7ec None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:28.300900 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe6fda-50ac-43c6-9230-538279bbe7ec None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:28.301384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe6fda-50ac-43c6-9230-538279bbe7ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:28.305247 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5265c9e0-af65-4525-8614-711277a5a1f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:28.306784 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5265c9e0-af65-4525-8614-711277a5a1f0 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:28.309133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe6fda-50ac-43c6-9230-538279bbe7ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:28.310795 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe6fda-50ac-43c6-9230-538279bbe7ec None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:28.321076 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-5265c9e0-af65-4525-8614-711277a5a1f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:28.321076 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:28.321076 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:28.321076 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:28.321076 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:28.321076 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:28.321076 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:28.321076 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:28.321076 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:28.321076 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:28.321076 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:28.321076 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:28.321076 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:28.321076 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:28.321076 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:28.322005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:28.322005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:28.322005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:28.322005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:28.322005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:28.322005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:28.322005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:28.322005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:28.322005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:28.322005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:28.322005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:28.322005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:28.322005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:28.322005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:28.322005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:28.322005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:28.322005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:28.322945 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:28.322945 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:28.322945 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:28.322945 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:28.322945 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:28.322945 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:28.322945 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:28.322945 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:28.327181 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 341/675] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:28.327879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-3afe6fda-50ac-43c6-9230-538279bbe7ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:28.327879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:28.327879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:28.327879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:28.327879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:28.327879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:28.327879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:28.327879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:28.327879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:28.327879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:28.327879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:28.327879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:28.327879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:28.327879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:28.327879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:28.328734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:28.328734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:28.328734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:28.328734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:28.328734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:28.328734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:28.328734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:28.328734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:28.328734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:28.328734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:28.328734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:28.328734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:28.328734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:28.328734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:28.328734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:28.328734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:28.328734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:28.329674 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:28.329674 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:28.329674 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:28.329674 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:28.329674 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:28.329674 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:28.329674 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:28.329674 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:28.330161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265c9e0-af65-4525-8614-711277a5a1f0 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:28.331411 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 335/676] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:28.332174 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe6fda-50ac-43c6-9230-538279bbe7ec None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:28.438354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe6fda-50ac-43c6-9230-538279bbe7ec None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:28.447020 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265c9e0-af65-4525-8614-711277a5a1f0 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:28.470625 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b06b1-cbb7-4475-b546-07d09546fd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:28.471282 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b06b1-cbb7-4475-b546-07d09546fd59 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:28.471558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b06b1-cbb7-4475-b546-07d09546fd59 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:28.471829 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b06b1-cbb7-4475-b546-07d09546fd59 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:28.472723 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b06b1-cbb7-4475-b546-07d09546fd59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:28.473034 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b06b1-cbb7-4475-b546-07d09546fd59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:28.474524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b06b1-cbb7-4475-b546-07d09546fd59 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:28.480809 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780803a2-9432-4f5a-91cf-a09db368d3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:28.481872 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780803a2-9432-4f5a-91cf-a09db368d3db None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:28.482345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780803a2-9432-4f5a-91cf-a09db368d3db None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:28.482760 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780803a2-9432-4f5a-91cf-a09db368d3db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:28.487045 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 336/677] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:28.487843 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b06b1-cbb7-4475-b546-07d09546fd59 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:28.496146 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780803a2-9432-4f5a-91cf-a09db368d3db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:28.497758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780803a2-9432-4f5a-91cf-a09db368d3db None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:28.515515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-780803a2-9432-4f5a-91cf-a09db368d3db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:28.515515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:28.515515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:28.515515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:28.515515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:28.515515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:28.515515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:28.515515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:28.515515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:28.515515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:28.515515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:28.515515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:28.515515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:28.515515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:28.515515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:28.516615 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:28.516615 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:28.516615 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:28.516615 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:28.516615 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:28.516615 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:28.516615 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:28.516615 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:28.516615 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:28.516615 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:28.516615 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:28.516615 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:28.516615 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:28.516615 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:28.516615 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:28.516615 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:28.516615 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:28.517787 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:28.517787 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:28.517787 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:28.517787 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:28.521004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 342/678] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:28.522321 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e615012-8680-4bb0-8429-e63d2c51a005 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:28.522600 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e615012-8680-4bb0-8429-e63d2c51a005 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:28.522893 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e615012-8680-4bb0-8429-e63d2c51a005 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:28.524129 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 343/679] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:28.524803 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e615012-8680-4bb0-8429-e63d2c51a005 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:28.592178 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b06b1-cbb7-4475-b546-07d09546fd59 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:28.627537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c9691-c91e-4a79-83fe-4864934abec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:28.628197 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c9691-c91e-4a79-83fe-4864934abec4 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:28.628470 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c9691-c91e-4a79-83fe-4864934abec4 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:28.628736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c9691-c91e-4a79-83fe-4864934abec4 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:28.629509 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c9691-c91e-4a79-83fe-4864934abec4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:28.629812 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c9691-c91e-4a79-83fe-4864934abec4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:28.631311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c9691-c91e-4a79-83fe-4864934abec4 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:28.638826 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e615012-8680-4bb0-8429-e63d2c51a005 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:28.647205 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 337/680] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:28.647205 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c9691-c91e-4a79-83fe-4864934abec4 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:28.672279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90205e8-a084-4240-ae7a-7a9274dd9b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:28.673082 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90205e8-a084-4240-ae7a-7a9274dd9b3e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:28.673364 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90205e8-a084-4240-ae7a-7a9274dd9b3e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:28.673685 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90205e8-a084-4240-ae7a-7a9274dd9b3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:28.674556 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90205e8-a084-4240-ae7a-7a9274dd9b3e None admin] RBAC: Adding query filter params (role.id=44847d50e6cb4bc7938c59b4e2d99833, scope.project.id=0500e378ea374ca692e230bbd8781318, user.id=96fd3f6dd9e74ba7a4ad159e43e8ae96) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:28.674860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90205e8-a084-4240-ae7a-7a9274dd9b3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:28.676327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90205e8-a084-4240-ae7a-7a9274dd9b3e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:28.690351 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 344/681] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:49:28 2022] GET /identity/v3/role_assignments?user.id=96fd3f6dd9e74ba7a4ad159e43e8ae96&scope.project.id=0500e378ea374ca692e230bbd8781318&role.id=44847d50e6cb4bc7938c59b4e2d99833 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:28.690351 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943175b-7a2a-44d4-a109-581b6890b8f0 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:28.690351 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943175b-7a2a-44d4-a109-581b6890b8f0 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:28.690351 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943175b-7a2a-44d4-a109-581b6890b8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:28.700900 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-f943175b-7a2a-44d4-a109-581b6890b8f0 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:28.702108 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f943175b-7a2a-44d4-a109-581b6890b8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:28.772852 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c9691-c91e-4a79-83fe-4864934abec4 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:28.774214 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 345/682] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:49:28.775177 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943175b-7a2a-44d4-a109-581b6890b8f0 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:28.835159 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eeac97-f85b-46a6-853a-4285464e9de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:28.835958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eeac97-f85b-46a6-853a-4285464e9de5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:28.836451 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eeac97-f85b-46a6-853a-4285464e9de5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:28.836822 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eeac97-f85b-46a6-853a-4285464e9de5 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:28.837991 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eeac97-f85b-46a6-853a-4285464e9de5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:28.838589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eeac97-f85b-46a6-853a-4285464e9de5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:28.840195 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eeac97-f85b-46a6-853a-4285464e9de5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:28.857812 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 338/683] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:28.859004 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eeac97-f85b-46a6-853a-4285464e9de5 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:28.898984 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943175b-7a2a-44d4-a109-581b6890b8f0 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:28.949665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d9e570-5631-4dce-8e35-77be033f65d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:28.950366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9e570-5631-4dce-8e35-77be033f65d0 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:28.950647 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9e570-5631-4dce-8e35-77be033f65d0 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:28.950920 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9e570-5631-4dce-8e35-77be033f65d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:28.951748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d9e570-5631-4dce-8e35-77be033f65d0 None admin] RBAC: Adding query filter params (role.id=44847d50e6cb4bc7938c59b4e2d99833, scope.project.id=0500e378ea374ca692e230bbd8781318, user.id=a81caf0558f2449cbbc8f4cd9446f363) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:28.952056 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d9e570-5631-4dce-8e35-77be033f65d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:28.953488 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d9e570-5631-4dce-8e35-77be033f65d0 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:28.959894 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 346/684] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:49:28 2022] GET /identity/v3/role_assignments?user.id=a81caf0558f2449cbbc8f4cd9446f363&scope.project.id=0500e378ea374ca692e230bbd8781318&role.id=44847d50e6cb4bc7938c59b4e2d99833 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:28.966231 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d9e570-5631-4dce-8e35-77be033f65d0 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:28.986229 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eeac97-f85b-46a6-853a-4285464e9de5 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:29.025475 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadcc7a-fb51-4495-b3d1-6a46bf2409aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:29.026350 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadcc7a-fb51-4495-b3d1-6a46bf2409aa None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:29.026628 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadcc7a-fb51-4495-b3d1-6a46bf2409aa None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:29.026944 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadcc7a-fb51-4495-b3d1-6a46bf2409aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:29.038145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aadcc7a-fb51-4495-b3d1-6a46bf2409aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:29.046236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-3aadcc7a-fb51-4495-b3d1-6a46bf2409aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:29.046236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:29.046236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:29.046236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:29.046236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:29.046236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:29.046236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:29.046236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:29.046236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:29.046236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:29.046236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:29.046236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:29.046236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:29.046236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:29.046236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:29.046236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:29.047192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:29.047192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:29.047192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:29.047192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:29.047192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:29.047192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:29.047192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:29.047192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:29.047192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:29.047192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:29.047192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:29.047192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:29.047192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:29.047192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:29.047192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:29.047192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:29.048067 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:29.048067 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:29.048067 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:29.048067 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:29.048067 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:29.048067 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:29.048067 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:29.048067 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:29.048067 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:29.048067 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:29.048067 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:29.048067 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:29.048067 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 339/685] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 9 22:49:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:29.054456 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadcc7a-fb51-4495-b3d1-6a46bf2409aa None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:29.097671 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d9e570-5631-4dce-8e35-77be033f65d0 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:29.140416 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f403bbb-5572-4ab7-8b32-a5ecab70fcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:29.141202 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f403bbb-5572-4ab7-8b32-a5ecab70fcbe None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:29.141484 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f403bbb-5572-4ab7-8b32-a5ecab70fcbe None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:29.141757 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f403bbb-5572-4ab7-8b32-a5ecab70fcbe None admin] PATH_INFO: `/v3/users/nova` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:29.150166 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f403bbb-5572-4ab7-8b32-a5ecab70fcbe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:29.151649 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f403bbb-5572-4ab7-8b32-a5ecab70fcbe None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:29.167378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-9f403bbb-5572-4ab7-8b32-a5ecab70fcbe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 22:49:29.167378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:29.167378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:29.167378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:29.167378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:29.167378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:29.167378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:29.167378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:29.167378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:29.167378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:29.167378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:29.167378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:29.167378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:29.167378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:29.167378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:29.167378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:29.169617 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:29.169617 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:29.169617 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:29.169617 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:29.169617 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:29.169617 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:29.169617 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:29.169617 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:29.169617 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:29.169617 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:29.169617 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:29.169617 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:29.169617 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:29.169617 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:29.169617 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:29.169617 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:29.169617 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:29.169617 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:29.171863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:29.171863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:29.171863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:29.171863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:29.171863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:29.171863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:29.171863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:29.171863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:29.171863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 22:49:29.171863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:29.171863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 347/686] 10.222.0.23 () {56 vars in 1093 bytes} [Mon May 9 22:49:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:29.173257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadcc7a-fb51-4495-b3d1-6a46bf2409aa None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:29.178240 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f403bbb-5572-4ab7-8b32-a5ecab70fcbe None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:29.245666 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe2b46-020f-424f-a02e-1da6325e3b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:29.246365 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe2b46-020f-424f-a02e-1da6325e3b57 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:29.246644 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe2b46-020f-424f-a02e-1da6325e3b57 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:29.246917 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe2b46-020f-424f-a02e-1da6325e3b57 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:29.247664 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe2b46-020f-424f-a02e-1da6325e3b57 None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:29.247965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe2b46-020f-424f-a02e-1da6325e3b57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:29.249332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe2b46-020f-424f-a02e-1da6325e3b57 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:29.258307 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 340/687] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 9 22:49:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:29.263690 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe2b46-020f-424f-a02e-1da6325e3b57 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:29.316751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f403bbb-5572-4ab7-8b32-a5ecab70fcbe None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:29.363864 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da901f7e-d370-4ed5-849a-c4b8d303875f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:29.364579 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da901f7e-d370-4ed5-849a-c4b8d303875f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:29.364861 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da901f7e-d370-4ed5-849a-c4b8d303875f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:29.365132 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da901f7e-d370-4ed5-849a-c4b8d303875f None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:29.365978 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da901f7e-d370-4ed5-849a-c4b8d303875f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:29.366303 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da901f7e-d370-4ed5-849a-c4b8d303875f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:29.367732 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da901f7e-d370-4ed5-849a-c4b8d303875f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:29.371326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe2b46-020f-424f-a02e-1da6325e3b57 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:29.398152 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 348/688] 10.222.0.23 () {56 vars in 1133 bytes} [Mon May 9 22:49:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:29.403431 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da901f7e-d370-4ed5-849a-c4b8d303875f None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:29.405787 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798495f-a6a4-448b-841d-fb98acdc83ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:29.406678 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798495f-a6a4-448b-841d-fb98acdc83ad None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:29.407108 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798495f-a6a4-448b-841d-fb98acdc83ad None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:29.407615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798495f-a6a4-448b-841d-fb98acdc83ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:29.421572 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3798495f-a6a4-448b-841d-fb98acdc83ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:29.423683 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3798495f-a6a4-448b-841d-fb98acdc83ad None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:29.445904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-3798495f-a6a4-448b-841d-fb98acdc83ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:29.445904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:29.445904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:29.445904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:29.445904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:29.445904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:29.445904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:29.445904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:29.445904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:29.445904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:29.445904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:29.445904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:29.445904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:29.445904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:29.445904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:29.447014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:29.447014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:29.447014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:29.447014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:29.447014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:29.447014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:29.447014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:29.447014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:29.447014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:29.447014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:29.447014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:29.447014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:29.447014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:29.447014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:29.447014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:29.447014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:29.447014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:29.448266 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:29.448266 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:29.448266 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:29.448266 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:29.448266 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 341/689] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:29.459618 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798495f-a6a4-448b-841d-fb98acdc83ad None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:29.532720 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da901f7e-d370-4ed5-849a-c4b8d303875f None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:29.573944 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fcf27b-ca90-4085-8604-0ce73d0392ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:29.574630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcf27b-ca90-4085-8604-0ce73d0392ec None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:29.574916 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcf27b-ca90-4085-8604-0ce73d0392ec None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:29.575176 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcf27b-ca90-4085-8604-0ce73d0392ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:29.610103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798495f-a6a4-448b-841d-fb98acdc83ad None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:29.661852 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3127e6e-5f9c-4e26-8c70-cfc31e48558b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:29.662477 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3127e6e-5f9c-4e26-8c70-cfc31e48558b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:29.662758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3127e6e-5f9c-4e26-8c70-cfc31e48558b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:29.663033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3127e6e-5f9c-4e26-8c70-cfc31e48558b None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:29.667056 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3127e6e-5f9c-4e26-8c70-cfc31e48558b None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:29.667373 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3127e6e-5f9c-4e26-8c70-cfc31e48558b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:29.668754 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3127e6e-5f9c-4e26-8c70-cfc31e48558b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:29.696889 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 342/690] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:29.702350 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3127e6e-5f9c-4e26-8c70-cfc31e48558b None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:29.753848 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fcf27b-ca90-4085-8604-0ce73d0392ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:29.755379 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fcf27b-ca90-4085-8604-0ce73d0392ec None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:29.843114 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3127e6e-5f9c-4e26-8c70-cfc31e48558b None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:29.884783 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ce6e7-5060-4f5b-a9d2-3f6a763314cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:29.887013 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ce6e7-5060-4f5b-a9d2-3f6a763314cd None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:29.888180 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ce6e7-5060-4f5b-a9d2-3f6a763314cd None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:29.889282 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ce6e7-5060-4f5b-a9d2-3f6a763314cd None admin] PATH_INFO: `/v3/users/swift` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:29.898411 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ce6e7-5060-4f5b-a9d2-3f6a763314cd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:29.900476 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ce6e7-5060-4f5b-a9d2-3f6a763314cd None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:29.930548 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-387ce6e7-5060-4f5b-a9d2-3f6a763314cd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 22:49:29.930548 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:29.930548 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:29.930548 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:29.930548 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:29.930548 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:29.930548 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:29.930548 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:29.930548 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:29.930548 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:29.930548 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:29.930548 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:29.930548 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:29.930548 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:29.930548 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:29.930548 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:29.931566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:29.931566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:29.931566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:29.931566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:29.931566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:29.931566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:29.931566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:29.931566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:29.931566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:29.931566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:29.931566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:29.931566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:29.931566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:29.931566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:29.931566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:29.931566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:29.931566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:29.931566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:29.932530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:29.932530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:29.932530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:29.932530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:29.932530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:29.932530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:29.932530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:29.932530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:29.932530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 22:49:29.932530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:29.934997 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 343/691] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:49:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:29.940783 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ce6e7-5060-4f5b-a9d2-3f6a763314cd None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:29.943030 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 349/692] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:49:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:49:29.951229 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fcf27b-ca90-4085-8604-0ce73d0392ec None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:30.046595 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ce6e7-5060-4f5b-a9d2-3f6a763314cd None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:30.082737 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beca297-1e21-42c4-9da4-d59ac5d5dcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:30.084301 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beca297-1e21-42c4-9da4-d59ac5d5dcab None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:30.084789 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beca297-1e21-42c4-9da4-d59ac5d5dcab None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:30.085264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beca297-1e21-42c4-9da4-d59ac5d5dcab None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:30.086341 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beca297-1e21-42c4-9da4-d59ac5d5dcab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:30.086853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beca297-1e21-42c4-9da4-d59ac5d5dcab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:30.088480 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beca297-1e21-42c4-9da4-d59ac5d5dcab None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:30.090163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fcf27b-ca90-4085-8604-0ce73d0392ec None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:30.107744 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 344/693] 10.222.0.23 () {56 vars in 1135 bytes} [Mon May 9 22:49:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:30.119109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beca297-1e21-42c4-9da4-d59ac5d5dcab None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:30.133423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20ad6c-9370-45d4-a9b2-3d1da76e0529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:30.134235 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20ad6c-9370-45d4-a9b2-3d1da76e0529 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:30.134512 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20ad6c-9370-45d4-a9b2-3d1da76e0529 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:30.134781 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20ad6c-9370-45d4-a9b2-3d1da76e0529 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:30.151766 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf20ad6c-9370-45d4-a9b2-3d1da76e0529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:30.153417 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf20ad6c-9370-45d4-a9b2-3d1da76e0529 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:30.184192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-cf20ad6c-9370-45d4-a9b2-3d1da76e0529 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:30.184192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:30.184192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:30.184192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:30.184192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:30.184192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:30.184192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:30.184192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:30.184192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:30.184192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:30.184192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:30.184192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:30.184192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:30.184192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:30.184192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:30.185109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:30.185109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:30.185109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:30.185109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:30.185109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:30.185109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:30.185109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:30.185109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:30.185109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:30.185109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:30.185109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:30.185109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:30.185109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:30.185109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:30.185109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:30.185109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:30.185109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:30.186639 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:30.186639 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:30.186639 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:30.186639 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:30.186639 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 350/694] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:30.186639 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316dbbb-4a90-4634-a99f-1540c94b1b29 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:30.187009 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316dbbb-4a90-4634-a99f-1540c94b1b29 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:30.187209 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316dbbb-4a90-4634-a99f-1540c94b1b29 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:30.188654 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 351/695] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:30.191356 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316dbbb-4a90-4634-a99f-1540c94b1b29 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:30.258053 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beca297-1e21-42c4-9da4-d59ac5d5dcab None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:30.291502 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201f3cd-a389-46fb-bdcf-432428d34278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:30.292210 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201f3cd-a389-46fb-bdcf-432428d34278 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:30.292489 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201f3cd-a389-46fb-bdcf-432428d34278 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:30.292759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201f3cd-a389-46fb-bdcf-432428d34278 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:30.320166 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316dbbb-4a90-4634-a99f-1540c94b1b29 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:30.365802 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af4fc0-370d-48db-8c89-c56c0aef831e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:30.366250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f201f3cd-a389-46fb-bdcf-432428d34278 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:30.366479 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af4fc0-370d-48db-8c89-c56c0aef831e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:30.366756 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af4fc0-370d-48db-8c89-c56c0aef831e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:30.367021 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af4fc0-370d-48db-8c89-c56c0aef831e None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:30.367725 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af4fc0-370d-48db-8c89-c56c0aef831e None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:30.368024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af4fc0-370d-48db-8c89-c56c0aef831e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:30.368325 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f201f3cd-a389-46fb-bdcf-432428d34278 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:30.369343 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af4fc0-370d-48db-8c89-c56c0aef831e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:30.386272 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 352/696] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:30.387540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa42cb5-f397-45d3-a96e-a9e68f6439a1 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:30.387823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa42cb5-f397-45d3-a96e-a9e68f6439a1 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:30.388137 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa42cb5-f397-45d3-a96e-a9e68f6439a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:30.409101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-3aa42cb5-f397-45d3-a96e-a9e68f6439a1 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:30.410287 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3aa42cb5-f397-45d3-a96e-a9e68f6439a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:30.474223 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 345/697] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:49:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 22:49:30.475026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201f3cd-a389-46fb-bdcf-432428d34278 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:30.489819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 353/698] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:49:30.490738 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa42cb5-f397-45d3-a96e-a9e68f6439a1 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:30.590546 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201f3cd-a389-46fb-bdcf-432428d34278 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:30.600279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa42cb5-f397-45d3-a96e-a9e68f6439a1 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:30.632945 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ebbe1-811b-4ec1-91fa-0a313902d69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:30.633739 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ebbe1-811b-4ec1-91fa-0a313902d69f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:30.634017 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ebbe1-811b-4ec1-91fa-0a313902d69f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:30.634319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ebbe1-811b-4ec1-91fa-0a313902d69f None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:30.639024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c31c27-8172-4721-ab89-75a0bf54be28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:30.639823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c31c27-8172-4721-ab89-75a0bf54be28 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:30.640096 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c31c27-8172-4721-ab89-75a0bf54be28 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:30.640360 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c31c27-8172-4721-ab89-75a0bf54be28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:30.644377 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ebbe1-811b-4ec1-91fa-0a313902d69f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:30.647403 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ebbe1-811b-4ec1-91fa-0a313902d69f None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:30.651719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c31c27-8172-4721-ab89-75a0bf54be28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:30.653139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c31c27-8172-4721-ab89-75a0bf54be28 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:30.659735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-dd0ebbe1-811b-4ec1-91fa-0a313902d69f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:30.659735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:30.659735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:30.659735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:30.659735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:30.659735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:30.659735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:30.659735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:30.659735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:30.659735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:30.659735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:30.659735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:30.659735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:30.659735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:30.659735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:30.661896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:30.661896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:30.661896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:30.661896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:30.661896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:30.661896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:30.661896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:30.661896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:30.661896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:30.661896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:30.661896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:30.661896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:30.661896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:30.661896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:30.661896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:30.661896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:30.661896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:30.664074 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:30.664074 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:30.664074 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:30.664074 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:30.664074 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:30.664074 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:30.664074 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:30.664074 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:30.664074 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 346/699] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:30.664074 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ebbe1-811b-4ec1-91fa-0a313902d69f None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:30.675064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-81c31c27-8172-4721-ab89-75a0bf54be28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:30.675064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:30.675064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:30.675064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:30.675064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:30.675064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:30.675064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:30.675064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:30.675064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:30.675064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:30.675064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:30.675064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:30.675064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:30.675064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:30.675064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:30.677658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:30.677658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:30.677658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:30.677658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:30.677658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:30.677658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:30.677658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:30.677658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:30.677658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:30.677658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:30.677658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:30.677658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:30.677658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:30.677658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:30.677658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:30.677658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:30.677658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:30.679950 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:30.679950 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:30.679950 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:30.679950 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:30.682144 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 354/700] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:30.682144 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c31c27-8172-4721-ab89-75a0bf54be28 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:30.768538 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ebbe1-811b-4ec1-91fa-0a313902d69f None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:30.780722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c31c27-8172-4721-ab89-75a0bf54be28 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:30.801600 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a290a1e-5816-4019-b594-8c81ef3a8e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:30.803062 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a290a1e-5816-4019-b594-8c81ef3a8e44 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:30.803342 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a290a1e-5816-4019-b594-8c81ef3a8e44 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:30.803616 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a290a1e-5816-4019-b594-8c81ef3a8e44 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:30.804259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a290a1e-5816-4019-b594-8c81ef3a8e44 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:30.805625 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a290a1e-5816-4019-b594-8c81ef3a8e44 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:30.815355 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b1b67-5807-4756-a08d-bfe7b756eeb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:30.816426 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b1b67-5807-4756-a08d-bfe7b756eeb7 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:30.816882 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b1b67-5807-4756-a08d-bfe7b756eeb7 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:30.818174 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b1b67-5807-4756-a08d-bfe7b756eeb7 None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:30.819095 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b1b67-5807-4756-a08d-bfe7b756eeb7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:30.819405 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b1b67-5807-4756-a08d-bfe7b756eeb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:30.820520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-3a290a1e-5816-4019-b594-8c81ef3a8e44 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 22:49:30.820520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:30.820520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:49:30.820520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:49:30.820520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:49:30.820520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:49:30.820520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:49:30.820520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:30.820520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 22:49:30.820520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:30.820520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:49:30.820520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:30.820520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:30.820520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:30.820520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:30.820520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:30.822719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:30.822719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:30.822719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:30.822719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:30.822719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:30.822719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:30.822719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:30.822719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:49:30.822719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:49:30.822719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:49:30.822719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:49:30.822719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:30.822719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:30.822719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:30.822719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:30.822719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:30.824154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:30.824154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:30.824154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:30.824154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:30.824154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:30.824154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:30.824154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:30.824154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:30.824154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:30.824154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:30.824154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:30.824154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:49:30.824154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:30.824154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 22:49:30.824154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:30.825116 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b1b67-5807-4756-a08d-bfe7b756eeb7 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:30.825202 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 347/701] 10.222.0.23 () {56 vars in 1129 bytes} [Mon May 9 22:49:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:30.825202 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a290a1e-5816-4019-b594-8c81ef3a8e44 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:30.839868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 355/702] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:30.840950 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b1b67-5807-4756-a08d-bfe7b756eeb7 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:30.935611 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a290a1e-5816-4019-b594-8c81ef3a8e44 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:30.958655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b1b67-5807-4756-a08d-bfe7b756eeb7 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:30.976382 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb97a4-dd4f-46fb-bf66-555f2a3e6700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:30.977041 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb97a4-dd4f-46fb-bf66-555f2a3e6700 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:30.977318 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb97a4-dd4f-46fb-bf66-555f2a3e6700 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:30.977587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb97a4-dd4f-46fb-bf66-555f2a3e6700 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:30.978733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb97a4-dd4f-46fb-bf66-555f2a3e6700 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:30.979036 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb97a4-dd4f-46fb-bf66-555f2a3e6700 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:30.980663 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb97a4-dd4f-46fb-bf66-555f2a3e6700 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:30.992754 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d133434-9b44-42e9-b434-121269fce271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:30.993582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d133434-9b44-42e9-b434-121269fce271 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:30.993956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d133434-9b44-42e9-b434-121269fce271 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:30.994368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d133434-9b44-42e9-b434-121269fce271 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:30.995159 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d133434-9b44-42e9-b434-121269fce271 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:30.995557 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d133434-9b44-42e9-b434-121269fce271 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:30.997082 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d133434-9b44-42e9-b434-121269fce271 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:30.998271 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 348/703] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:30.999141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb97a4-dd4f-46fb-bf66-555f2a3e6700 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:31.003714 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 356/704] 10.222.0.23 () {56 vars in 1124 bytes} [Mon May 9 22:49:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:31.004680 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d133434-9b44-42e9-b434-121269fce271 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:31.107908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb97a4-dd4f-46fb-bf66-555f2a3e6700 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:31.133970 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d133434-9b44-42e9-b434-121269fce271 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:31.142154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4dc1ed-4b0a-4f25-8ead-bd0bd7697ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:31.143000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4dc1ed-4b0a-4f25-8ead-bd0bd7697ec1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:31.143378 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4dc1ed-4b0a-4f25-8ead-bd0bd7697ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:31.143748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4dc1ed-4b0a-4f25-8ead-bd0bd7697ec1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:31.144672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4dc1ed-4b0a-4f25-8ead-bd0bd7697ec1 None admin] RBAC: Adding query filter params (role.id=bda550492f4f4c5c83f52fa89fc18a7d, scope.project.id=0500e378ea374ca692e230bbd8781318, user.id=02fcd7cc86134ce488c7dd75dfb174b8) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:31.145105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4dc1ed-4b0a-4f25-8ead-bd0bd7697ec1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:31.146759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4dc1ed-4b0a-4f25-8ead-bd0bd7697ec1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:31.152915 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 349/705] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:49:30 2022] GET /identity/v3/role_assignments?user.id=02fcd7cc86134ce488c7dd75dfb174b8&scope.project.id=0500e378ea374ca692e230bbd8781318&role.id=bda550492f4f4c5c83f52fa89fc18a7d => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:31.153810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4dc1ed-4b0a-4f25-8ead-bd0bd7697ec1 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:31.195157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825c7c3-3e09-4031-a506-aab94582ccfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:31.195965 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825c7c3-3e09-4031-a506-aab94582ccfa None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:31.197399 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825c7c3-3e09-4031-a506-aab94582ccfa None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:31.197673 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825c7c3-3e09-4031-a506-aab94582ccfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:31.211968 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825c7c3-3e09-4031-a506-aab94582ccfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:31.213423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825c7c3-3e09-4031-a506-aab94582ccfa None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:31.235227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-f825c7c3-3e09-4031-a506-aab94582ccfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:31.235227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:31.235227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:31.235227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:31.235227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:31.235227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:31.235227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:31.235227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:31.235227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:31.235227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:31.235227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:31.235227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:31.235227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:31.235227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:31.235227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:31.236207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:31.236207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:31.236207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:31.236207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:31.236207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:31.236207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:31.236207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:31.236207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:31.236207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:31.236207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:31.236207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:31.236207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:31.236207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:31.236207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:31.236207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:31.236207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:31.236207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:31.237116 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:31.237116 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:31.237116 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:31.237116 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:31.237116 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:31.237116 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:31.237116 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:31.237116 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:31.237116 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 357/706] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:31.238070 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677269b-6efa-4ebe-be26-d435c783753e None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:31.238365 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677269b-6efa-4ebe-be26-d435c783753e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:31.238639 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677269b-6efa-4ebe-be26-d435c783753e None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:31.243917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 358/707] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:31.243917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677269b-6efa-4ebe-be26-d435c783753e None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:31.306744 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4dc1ed-4b0a-4f25-8ead-bd0bd7697ec1 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:31.345131 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37ac39-973e-41de-aadd-a12448d79533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:31.346213 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37ac39-973e-41de-aadd-a12448d79533 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:31.346745 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37ac39-973e-41de-aadd-a12448d79533 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:31.347227 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37ac39-973e-41de-aadd-a12448d79533 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:31.348143 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab37ac39-973e-41de-aadd-a12448d79533 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:31.349216 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab37ac39-973e-41de-aadd-a12448d79533 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:31.350865 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab37ac39-973e-41de-aadd-a12448d79533 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:31.352500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677269b-6efa-4ebe-be26-d435c783753e None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:31.358142 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 350/708] 10.222.0.23 () {56 vars in 1124 bytes} [Mon May 9 22:49:31 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:31.359722 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa07cdc-2c2e-44b4-86c5-e07da92888ce None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:31.360075 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa07cdc-2c2e-44b4-86c5-e07da92888ce None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:31.360422 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa07cdc-2c2e-44b4-86c5-e07da92888ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:31.378061 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-2aa07cdc-2c2e-44b4-86c5-e07da92888ce None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:31.379253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-2aa07cdc-2c2e-44b4-86c5-e07da92888ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:31.393879 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f06159-13e2-4e90-b510-3c627f43440a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:31.394937 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f06159-13e2-4e90-b510-3c627f43440a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:31.395956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f06159-13e2-4e90-b510-3c627f43440a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:31.396441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f06159-13e2-4e90-b510-3c627f43440a None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:31.397490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f06159-13e2-4e90-b510-3c627f43440a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:31.397992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f06159-13e2-4e90-b510-3c627f43440a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:31.399720 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f06159-13e2-4e90-b510-3c627f43440a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:31.413785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 359/709] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:31.420911 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f06159-13e2-4e90-b510-3c627f43440a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:31.452651 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 351/710] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:49:31.475371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa07cdc-2c2e-44b4-86c5-e07da92888ce None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:31.584834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f06159-13e2-4e90-b510-3c627f43440a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:31.615382 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa07cdc-2c2e-44b4-86c5-e07da92888ce None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:31.640074 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d0b82-f8ee-43bb-8f45-22992fd8c8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:31.640738 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d0b82-f8ee-43bb-8f45-22992fd8c8fd None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:31.641017 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d0b82-f8ee-43bb-8f45-22992fd8c8fd None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:31.641312 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d0b82-f8ee-43bb-8f45-22992fd8c8fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:31.642140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d0b82-f8ee-43bb-8f45-22992fd8c8fd None admin] RBAC: Adding query filter params (role.id=44847d50e6cb4bc7938c59b4e2d99833, scope.project.id=0500e378ea374ca692e230bbd8781318, user.id=8cb6c294ce1842b68a6bc73ac381ed6b) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:31.642496 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d0b82-f8ee-43bb-8f45-22992fd8c8fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:31.643935 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d0b82-f8ee-43bb-8f45-22992fd8c8fd None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:31.658179 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 360/711] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:49:31 2022] GET /identity/v3/role_assignments?user.id=8cb6c294ce1842b68a6bc73ac381ed6b&scope.project.id=0500e378ea374ca692e230bbd8781318&role.id=44847d50e6cb4bc7938c59b4e2d99833 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:31.668231 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2d667-134d-4444-892c-1065edbf0ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:31.669044 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2d667-134d-4444-892c-1065edbf0ead None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:31.669325 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2d667-134d-4444-892c-1065edbf0ead None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:31.669599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2d667-134d-4444-892c-1065edbf0ead None admin] PATH_INFO: `/v3/services/network` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:31.670308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c2d667-134d-4444-892c-1065edbf0ead None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:31.671701 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c2d667-134d-4444-892c-1065edbf0ead None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:31.692921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-31c2d667-134d-4444-892c-1065edbf0ead None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 09 22:49:31.692921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:31.692921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:49:31.692921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:49:31.692921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:49:31.692921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:49:31.692921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:49:31.692921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:31.692921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 22:49:31.692921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:31.692921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:49:31.692921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:31.692921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:31.692921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:31.692921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:31.692921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:31.692921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:31.694025 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:31.694025 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:31.694025 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:31.694025 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:31.694025 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:31.694025 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:31.694025 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:49:31.694025 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:49:31.694025 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:49:31.694025 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:49:31.694025 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:31.694025 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:31.694025 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:31.694025 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:31.694025 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:31.694025 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:31.694968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:31.694968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:31.694968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:31.694968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:31.694968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:31.694968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:31.694968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:31.694968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:31.694968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:31.694968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:31.694968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:49:31.694968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:31.694968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 22:49:31.694968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:31.694968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 352/712] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:31 2022] GET /identity/v3/services/network => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:31.701926 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d0b82-f8ee-43bb-8f45-22992fd8c8fd None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:31.818848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166e2bb-dcde-4505-9603-e9e06e9efc0d None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:31.819221 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166e2bb-dcde-4505-9603-e9e06e9efc0d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:31.819421 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166e2bb-dcde-4505-9603-e9e06e9efc0d None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:31.820887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 353/713] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:49:31.828495 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c37a52-0008-4df6-bf4d-3408ebddd514 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:31.828806 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c37a52-0008-4df6-bf4d-3408ebddd514 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:31.829049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c37a52-0008-4df6-bf4d-3408ebddd514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:31.845988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d0b82-f8ee-43bb-8f45-22992fd8c8fd None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:31.852890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e6c37a52-0008-4df6-bf4d-3408ebddd514 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:31.854102 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e6c37a52-0008-4df6-bf4d-3408ebddd514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:31.903967 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf33217e-9012-4570-a11b-bc61507875cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:31.905053 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33217e-9012-4570-a11b-bc61507875cf None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:31.905340 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33217e-9012-4570-a11b-bc61507875cf None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:31.905616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33217e-9012-4570-a11b-bc61507875cf None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:31.906365 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf33217e-9012-4570-a11b-bc61507875cf None admin] RBAC: Adding query filter params (name=network) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:31.906674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf33217e-9012-4570-a11b-bc61507875cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:31.908389 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf33217e-9012-4570-a11b-bc61507875cf None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:31.920439 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 361/714] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 9 22:49:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:31.922354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf33217e-9012-4570-a11b-bc61507875cf None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:31.940558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 354/715] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:49:31.971381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c37a52-0008-4df6-bf4d-3408ebddd514 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:32.057886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf33217e-9012-4570-a11b-bc61507875cf None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:32.101349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d8f15-5516-4fef-bf1e-973903503b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:32.102415 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d8f15-5516-4fef-bf1e-973903503b8b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:32.102921 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d8f15-5516-4fef-bf1e-973903503b8b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:32.103393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d8f15-5516-4fef-bf1e-973903503b8b None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:32.103499 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c37a52-0008-4df6-bf4d-3408ebddd514 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:32.104391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d8f15-5516-4fef-bf1e-973903503b8b None admin] RBAC: Adding query filter params (type=network) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:32.104894 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d8f15-5516-4fef-bf1e-973903503b8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:32.106548 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d8f15-5516-4fef-bf1e-973903503b8b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:32.112966 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 362/716] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 9 22:49:31 2022] GET /identity/v3/services?type=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:32.144322 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0905d2e-7c87-4c62-af96-3be9f4528b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:32.145128 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0905d2e-7c87-4c62-af96-3be9f4528b37 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:32.146062 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0905d2e-7c87-4c62-af96-3be9f4528b37 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:32.146389 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0905d2e-7c87-4c62-af96-3be9f4528b37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:32.158172 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0905d2e-7c87-4c62-af96-3be9f4528b37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:32.159596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0905d2e-7c87-4c62-af96-3be9f4528b37 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:32.179428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-a0905d2e-7c87-4c62-af96-3be9f4528b37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:32.179428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:32.179428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:32.179428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:32.179428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:32.179428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:32.179428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:32.179428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:32.179428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:32.179428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:32.179428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:32.179428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:32.179428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:32.179428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:32.179428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:32.180420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:32.180420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:32.180420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:32.180420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:32.180420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:32.180420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:32.180420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:32.180420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:32.180420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:32.180420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:32.180420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:32.180420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:32.180420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:32.180420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:32.180420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:32.180420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:32.180420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:32.181340 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:32.181340 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:32.181340 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:32.181340 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:32.181340 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 355/717] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:32.187813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d8f15-5516-4fef-bf1e-973903503b8b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:32.316568 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d8f15-5516-4fef-bf1e-973903503b8b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:32.360265 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c77e37-615a-4210-aee2-2200b07f9719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:32.361313 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c77e37-615a-4210-aee2-2200b07f9719 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:32.361793 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c77e37-615a-4210-aee2-2200b07f9719 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:32.362333 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c77e37-615a-4210-aee2-2200b07f9719 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:32.363306 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c77e37-615a-4210-aee2-2200b07f9719 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:32.363877 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c77e37-615a-4210-aee2-2200b07f9719 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:32.365480 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c77e37-615a-4210-aee2-2200b07f9719 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:32.378785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 363/718] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:32.385123 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0905d2e-7c87-4c62-af96-3be9f4528b37 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:32.495298 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0905d2e-7c87-4c62-af96-3be9f4528b37 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:32.541147 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2272c5f-8923-4702-a80b-1bf6383d9e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:32.542239 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2272c5f-8923-4702-a80b-1bf6383d9e4c None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:32.542725 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2272c5f-8923-4702-a80b-1bf6383d9e4c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:32.543200 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2272c5f-8923-4702-a80b-1bf6383d9e4c None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:32.544127 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2272c5f-8923-4702-a80b-1bf6383d9e4c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:32.545841 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2272c5f-8923-4702-a80b-1bf6383d9e4c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:32.589589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 356/719] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 9 22:49:32 2022] POST /identity/v3/users => generated 254 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:49:33.830569 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a8886c-f2db-4fce-8edc-3b5cec3004d8 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:33.831614 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a8886c-f2db-4fce-8edc-3b5cec3004d8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:33.832180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a8886c-f2db-4fce-8edc-3b5cec3004d8 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:33.834641 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 364/720] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:33.840399 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce94daf-caf2-4f1a-8aec-f5132a9b89b5 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:33.841412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce94daf-caf2-4f1a-8aec-f5132a9b89b5 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:33.841895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce94daf-caf2-4f1a-8aec-f5132a9b89b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:33.858777 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-4ce94daf-caf2-4f1a-8aec-f5132a9b89b5 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:33.860503 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4ce94daf-caf2-4f1a-8aec-f5132a9b89b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:33.919185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ad8f5-1f0f-4e24-9a5b-8201cf704909 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:33.919778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ad8f5-1f0f-4e24-9a5b-8201cf704909 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:33.920285 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ad8f5-1f0f-4e24-9a5b-8201cf704909 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:33.921837 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 365/721] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:49:33.937290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07aedd-4dae-401d-ac37-e359afc96c26 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:33.937878 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07aedd-4dae-401d-ac37-e359afc96c26 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:33.938468 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07aedd-4dae-401d-ac37-e359afc96c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:33.939566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 357/722] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:49:33.971049 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ad07aedd-4dae-401d-ac37-e359afc96c26 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:33.972539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ad07aedd-4dae-401d-ac37-e359afc96c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:33.990991 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce94daf-caf2-4f1a-8aec-f5132a9b89b5 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:34.060625 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 366/723] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:49:34.109457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad07aedd-4dae-401d-ac37-e359afc96c26 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:34.132702 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce94daf-caf2-4f1a-8aec-f5132a9b89b5 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:34.169503 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf6f63-f2b3-4d16-8885-f1bb8b43aa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:34.170933 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf6f63-f2b3-4d16-8885-f1bb8b43aa7c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:34.171395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf6f63-f2b3-4d16-8885-f1bb8b43aa7c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:34.171975 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf6f63-f2b3-4d16-8885-f1bb8b43aa7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:34.183859 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf6f63-f2b3-4d16-8885-f1bb8b43aa7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:34.193528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-cddf6f63-f2b3-4d16-8885-f1bb8b43aa7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:49:34.193528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:34.193528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:34.193528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:34.193528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:34.193528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:34.193528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:34.193528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:34.193528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:34.193528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:34.193528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:34.193528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:34.193528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:34.193528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:34.193528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:34.193528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:34.194516 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:34.194516 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:34.194516 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:34.194516 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:34.194516 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:34.194516 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:34.194516 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:34.194516 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:34.194516 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:34.194516 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:34.194516 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:34.194516 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:34.194516 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:34.194516 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:34.194516 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:34.194516 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:34.195460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:34.195460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:34.195460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:34.195460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:34.195460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:34.195460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:34.195460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:34.195460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:34.195460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:34.195460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:34.195460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:49:34.195460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:34.198909 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 358/724] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:49:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:34.207185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf6f63-f2b3-4d16-8885-f1bb8b43aa7c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:34.316021 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad07aedd-4dae-401d-ac37-e359afc96c26 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:34.343817 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf6f63-f2b3-4d16-8885-f1bb8b43aa7c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:34.373429 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965a66b-0a02-417c-b018-25441a1aa7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:34.374096 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965a66b-0a02-417c-b018-25441a1aa7ad None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:34.374399 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965a66b-0a02-417c-b018-25441a1aa7ad None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:34.374668 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965a66b-0a02-417c-b018-25441a1aa7ad None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:34.375384 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965a66b-0a02-417c-b018-25441a1aa7ad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:34.376732 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965a66b-0a02-417c-b018-25441a1aa7ad None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:34.381818 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdd031-8d4f-4499-910a-8782a064a44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:34.382817 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdd031-8d4f-4499-910a-8782a064a44c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:34.383315 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdd031-8d4f-4499-910a-8782a064a44c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:34.383788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdd031-8d4f-4499-910a-8782a064a44c None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:34.384728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bdd031-8d4f-4499-910a-8782a064a44c None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:34.385231 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bdd031-8d4f-4499-910a-8782a064a44c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:34.386856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bdd031-8d4f-4499-910a-8782a064a44c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:34.396896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 359/725] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:49:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:34.402702 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdd031-8d4f-4499-910a-8782a064a44c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:34.423801 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 367/726] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 9 22:49:34 2022] POST /identity/v3/services => generated 251 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 22:49:34.510569 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdd031-8d4f-4499-910a-8782a064a44c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:34.561999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65aa711-b0a4-440f-8441-e2f1b5c22b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:34.563216 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65aa711-b0a4-440f-8441-e2f1b5c22b30 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:34.563755 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65aa711-b0a4-440f-8441-e2f1b5c22b30 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:34.564224 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65aa711-b0a4-440f-8441-e2f1b5c22b30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:34.576217 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65aa711-b0a4-440f-8441-e2f1b5c22b30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:34.578079 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65aa711-b0a4-440f-8441-e2f1b5c22b30 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:34.596804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-c65aa711-b0a4-440f-8441-e2f1b5c22b30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:34.596804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:34.596804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:34.596804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:34.596804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:34.596804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:34.596804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:34.596804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:34.596804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:34.596804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:34.596804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:34.596804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:34.596804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:34.596804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:34.596804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:34.597772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:34.597772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:34.597772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:34.597772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:34.597772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:34.597772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:34.597772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:34.597772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:34.597772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:34.597772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:34.597772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:34.597772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:34.597772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:34.597772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:34.597772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:34.597772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:34.597772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:34.599288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:34.599288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:34.599288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:34.599288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:34.601748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 360/727] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:34.611658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965a66b-0a02-417c-b018-25441a1aa7ad None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:34.771289 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965a66b-0a02-417c-b018-25441a1aa7ad None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:34.818843 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f87b5-0556-4143-afd1-67c80e53f594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:34.819526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f87b5-0556-4143-afd1-67c80e53f594 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:34.819905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f87b5-0556-4143-afd1-67c80e53f594 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:34.820178 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f87b5-0556-4143-afd1-67c80e53f594 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:34.820906 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f87b5-0556-4143-afd1-67c80e53f594 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:34.821207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f87b5-0556-4143-afd1-67c80e53f594 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:34.823904 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f87b5-0556-4143-afd1-67c80e53f594 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:34.840350 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef4eb9-1fc2-4df8-840e-65ba3a05963a None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:34.840990 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef4eb9-1fc2-4df8-840e-65ba3a05963a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:34.841529 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 368/728] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:34.842455 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef4eb9-1fc2-4df8-840e-65ba3a05963a None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:34.844251 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 361/729] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:34.847842 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f87b5-0556-4143-afd1-67c80e53f594 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:34.850047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b06f84-d835-4990-ab70-027524993924 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:34.850388 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b06f84-d835-4990-ab70-027524993924 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:34.850674 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b06f84-d835-4990-ab70-027524993924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:34.874224 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-64b06f84-d835-4990-ab70-027524993924 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:34.875795 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-64b06f84-d835-4990-ab70-027524993924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:34.973245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f87b5-0556-4143-afd1-67c80e53f594 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:34.980694 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 362/730] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:34 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 22:49:34.996524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abffaa-45a4-4a2b-b1ff-1b3802b45540 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:34.996802 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abffaa-45a4-4a2b-b1ff-1b3802b45540 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:34.997081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abffaa-45a4-4a2b-b1ff-1b3802b45540 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:34.998710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 363/731] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:49:35.003454 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bad676-7738-4106-bc20-97c96aaa40f5 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:35.003742 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bad676-7738-4106-bc20-97c96aaa40f5 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:35.004012 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bad676-7738-4106-bc20-97c96aaa40f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:35.029061 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-19bad676-7738-4106-bc20-97c96aaa40f5 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:35.030562 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4ba19-853e-4836-8241-1a9eefcb1978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:35.031152 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-19bad676-7738-4106-bc20-97c96aaa40f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:35.031408 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4ba19-853e-4836-8241-1a9eefcb1978 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:35.031734 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4ba19-853e-4836-8241-1a9eefcb1978 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:35.032013 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4ba19-853e-4836-8241-1a9eefcb1978 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:35.040335 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a4ba19-853e-4836-8241-1a9eefcb1978 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:35.041817 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a4ba19-853e-4836-8241-1a9eefcb1978 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:35.058019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-85a4ba19-853e-4836-8241-1a9eefcb1978 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 22:49:35.058019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:35.058019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:35.058019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:35.058019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:35.058019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:35.058019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:35.058019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:35.058019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:35.058019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:35.058019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:35.058019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:35.058019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:35.058019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:35.058019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:35.058019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:35.059025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:35.059025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:35.059025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:35.059025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:35.059025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:35.059025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:35.059025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:35.059025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:35.059025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:35.059025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:35.059025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:35.059025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:35.059025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:35.059025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:35.059025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:35.059025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:35.059025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:35.059025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:35.059970 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:35.059970 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:35.059970 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:35.059970 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:35.059970 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:35.059970 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:35.059970 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:35.059970 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:35.059970 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 22:49:35.059970 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:35.059970 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 369/732] 10.222.0.23 () {56 vars in 1093 bytes} [Mon May 9 22:49:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:35.060568 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4ba19-853e-4836-8241-1a9eefcb1978 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:35.113731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 364/733] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:35 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 22:49:35.115309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bad676-7738-4106-bc20-97c96aaa40f5 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:35.191757 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4ba19-853e-4836-8241-1a9eefcb1978 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:35.220105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bad676-7738-4106-bc20-97c96aaa40f5 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:35.226544 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9810a90-7312-42c4-9973-b1dc35a42974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:35.227211 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9810a90-7312-42c4-9973-b1dc35a42974 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:35.227488 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9810a90-7312-42c4-9973-b1dc35a42974 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:35.227757 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9810a90-7312-42c4-9973-b1dc35a42974 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:35.228476 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9810a90-7312-42c4-9973-b1dc35a42974 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:35.229851 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9810a90-7312-42c4-9973-b1dc35a42974 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:35.248903 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 370/734] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 9 22:49:35 2022] POST /identity/v3/services => generated 240 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:49:35.249786 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9810a90-7312-42c4-9973-b1dc35a42974 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:35.283162 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c47296-6fc1-4ac4-8495-813a1217c739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:35.284507 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c47296-6fc1-4ac4-8495-813a1217c739 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:35.285019 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c47296-6fc1-4ac4-8495-813a1217c739 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:35.285672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c47296-6fc1-4ac4-8495-813a1217c739 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:35.287181 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c47296-6fc1-4ac4-8495-813a1217c739 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:35.287716 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c47296-6fc1-4ac4-8495-813a1217c739 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:35.289855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c47296-6fc1-4ac4-8495-813a1217c739 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:35.331982 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 365/735] 10.222.0.23 () {56 vars in 1133 bytes} [Mon May 9 22:49:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:35.337796 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c47296-6fc1-4ac4-8495-813a1217c739 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:35.463039 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c47296-6fc1-4ac4-8495-813a1217c739 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:35.473362 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9810a90-7312-42c4-9973-b1dc35a42974 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:35.516298 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fb620-cb64-4ec3-a635-0b8ffd556243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:35.517100 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fb620-cb64-4ec3-a635-0b8ffd556243 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:35.517382 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fb620-cb64-4ec3-a635-0b8ffd556243 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:35.517655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fb620-cb64-4ec3-a635-0b8ffd556243 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:35.523349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108ccd2-e23f-4a66-b80b-b2b79a1a8b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:35.524395 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108ccd2-e23f-4a66-b80b-b2b79a1a8b38 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:35.525321 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108ccd2-e23f-4a66-b80b-b2b79a1a8b38 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:35.525793 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108ccd2-e23f-4a66-b80b-b2b79a1a8b38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:35.528357 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8fb620-cb64-4ec3-a635-0b8ffd556243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:35.530365 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8fb620-cb64-4ec3-a635-0b8ffd556243 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:35.540985 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108ccd2-e23f-4a66-b80b-b2b79a1a8b38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:35.549258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-3108ccd2-e23f-4a66-b80b-b2b79a1a8b38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:49:35.549258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:35.549258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:35.549258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:35.549258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:35.549258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:35.549258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:35.549258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:35.549258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:35.549258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:35.549258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:35.549258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:35.549258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:35.549258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:35.549258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:35.549258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:35.550515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:35.550515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:35.550515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:35.550515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:35.550515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:35.550515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:35.550515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:35.550515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:35.550515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:35.550515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:35.550515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:35.550515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:35.550515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:35.550515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:35.550515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:35.550515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:35.551425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:35.551425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:35.551425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:35.551425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:35.551425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:35.551425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:35.551425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:35.551425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:35.551425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:35.551425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:35.551425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:49:35.551425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:35.552627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-ab8fb620-cb64-4ec3-a635-0b8ffd556243 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:35.552627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:35.552627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:35.552627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:35.552627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:35.552627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:35.552627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:35.552627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:35.552627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:35.552627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:35.552627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:35.552627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:35.552627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:35.552627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:35.552627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:35.553435 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:35.553435 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:35.553435 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:35.553435 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:35.553435 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:35.553435 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:35.553435 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:35.553435 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:35.553435 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:35.553435 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:35.553435 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:35.553435 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:35.553435 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:35.553435 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:35.553435 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:35.553435 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:35.553435 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:35.555976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:35.555976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:35.555976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:35.555976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:35.555976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 366/736] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:35.556530 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 371/737] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:49:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:35.559063 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fb620-cb64-4ec3-a635-0b8ffd556243 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:35.564146 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108ccd2-e23f-4a66-b80b-b2b79a1a8b38 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:35.681716 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fb620-cb64-4ec3-a635-0b8ffd556243 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:35.687727 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108ccd2-e23f-4a66-b80b-b2b79a1a8b38 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:35.720729 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81c5f1-869e-4e2a-b9fc-8766abde401f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:35.721552 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81c5f1-869e-4e2a-b9fc-8766abde401f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:35.721838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81c5f1-869e-4e2a-b9fc-8766abde401f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:35.722421 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81c5f1-869e-4e2a-b9fc-8766abde401f None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:35.723150 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d81c5f1-869e-4e2a-b9fc-8766abde401f None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:35.723514 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d81c5f1-869e-4e2a-b9fc-8766abde401f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:35.725386 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d81c5f1-869e-4e2a-b9fc-8766abde401f None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:35.742557 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 367/738] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:35.744079 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81c5f1-869e-4e2a-b9fc-8766abde401f None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:35.767791 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480fa91-2570-467c-8693-4e9ecd22c571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:35.768514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480fa91-2570-467c-8693-4e9ecd22c571 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:35.768787 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480fa91-2570-467c-8693-4e9ecd22c571 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:35.769102 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480fa91-2570-467c-8693-4e9ecd22c571 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:35.769846 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480fa91-2570-467c-8693-4e9ecd22c571 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:35.770158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480fa91-2570-467c-8693-4e9ecd22c571 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:35.772085 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480fa91-2570-467c-8693-4e9ecd22c571 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:35.793720 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 372/739] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:49:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:35.798980 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480fa91-2570-467c-8693-4e9ecd22c571 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:35.855665 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81c5f1-869e-4e2a-b9fc-8766abde401f None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:35.889144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ec618-f943-413e-b573-aa63d7447074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:35.889944 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ec618-f943-413e-b573-aa63d7447074 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:35.890257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ec618-f943-413e-b573-aa63d7447074 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:35.890534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ec618-f943-413e-b573-aa63d7447074 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:35.898237 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ec618-f943-413e-b573-aa63d7447074 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:35.899687 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ec618-f943-413e-b573-aa63d7447074 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:35.914646 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-e64ec618-f943-413e-b573-aa63d7447074 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:35.914646 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:35.914646 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:35.914646 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:35.914646 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:35.914646 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:35.914646 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:35.914646 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:35.914646 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:35.914646 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:35.914646 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:35.914646 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:35.914646 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:35.914646 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:35.914646 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:35.915576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:35.915576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:35.915576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:35.915576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:35.915576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:35.915576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:35.915576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:35.915576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:35.915576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:35.915576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:35.915576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:35.915576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:35.915576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:35.915576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:35.915576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:35.915576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:35.915576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:35.916477 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:35.916477 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:35.916477 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:35.916477 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:35.916477 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:35.916477 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:35.916477 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:35.916477 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:35.920437 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 368/740] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:35.921824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3e696-49b7-4350-b971-8b194b9b47f6 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:35.922069 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3e696-49b7-4350-b971-8b194b9b47f6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:35.922371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3e696-49b7-4350-b971-8b194b9b47f6 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:35.928227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480fa91-2570-467c-8693-4e9ecd22c571 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:35.928696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 369/741] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:35.928696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3e696-49b7-4350-b971-8b194b9b47f6 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:35.966883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f991ffe-dae0-44ad-83eb-91ebf17c3697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:35.968085 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f991ffe-dae0-44ad-83eb-91ebf17c3697 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:35.968894 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f991ffe-dae0-44ad-83eb-91ebf17c3697 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:35.969648 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f991ffe-dae0-44ad-83eb-91ebf17c3697 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:35.981625 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f991ffe-dae0-44ad-83eb-91ebf17c3697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:35.983530 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f991ffe-dae0-44ad-83eb-91ebf17c3697 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:36.007354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-2f991ffe-dae0-44ad-83eb-91ebf17c3697 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:36.007354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:36.007354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:36.007354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:36.007354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:36.007354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:36.007354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:36.007354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:36.007354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:36.007354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:36.007354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:36.007354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:36.007354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:36.007354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:36.007354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:36.008372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:36.008372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:36.008372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:36.008372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:36.008372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:36.008372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:36.008372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:36.008372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:36.008372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:36.008372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:36.008372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:36.008372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:36.008372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:36.008372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:36.008372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:36.008372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:36.008372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:36.009297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:36.009297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:36.009297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:36.009297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:36.016089 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 373/742] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:36.017774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ee0e9-1956-45c1-ac0a-5ac04316b146 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:36.018282 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ee0e9-1956-45c1-ac0a-5ac04316b146 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:36.018777 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ee0e9-1956-45c1-ac0a-5ac04316b146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:36.037141 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d52ee0e9-1956-45c1-ac0a-5ac04316b146 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:36.038293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3e696-49b7-4350-b971-8b194b9b47f6 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:36.038922 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d52ee0e9-1956-45c1-ac0a-5ac04316b146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:36.075157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109365a0-196f-40e6-9203-6e0a60382147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:36.075746 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109365a0-196f-40e6-9203-6e0a60382147 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:36.076034 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109365a0-196f-40e6-9203-6e0a60382147 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:36.076312 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109365a0-196f-40e6-9203-6e0a60382147 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:36.077104 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109365a0-196f-40e6-9203-6e0a60382147 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:36.077415 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109365a0-196f-40e6-9203-6e0a60382147 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:36.079551 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109365a0-196f-40e6-9203-6e0a60382147 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:36.097328 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 370/743] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:36.098235 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109365a0-196f-40e6-9203-6e0a60382147 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:36.114485 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 374/744] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:49:36.115715 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ee0e9-1956-45c1-ac0a-5ac04316b146 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:36.212951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109365a0-196f-40e6-9203-6e0a60382147 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:36.242853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ee0e9-1956-45c1-ac0a-5ac04316b146 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:36.249215 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56ca5b-330b-49a2-b8eb-48a4a3d5417c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:36.249926 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56ca5b-330b-49a2-b8eb-48a4a3d5417c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:36.250231 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56ca5b-330b-49a2-b8eb-48a4a3d5417c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:36.250509 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56ca5b-330b-49a2-b8eb-48a4a3d5417c None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:36.251227 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56ca5b-330b-49a2-b8eb-48a4a3d5417c None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:36.251944 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56ca5b-330b-49a2-b8eb-48a4a3d5417c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:36.253646 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56ca5b-330b-49a2-b8eb-48a4a3d5417c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:36.270013 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 371/745] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:36.274433 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56ca5b-330b-49a2-b8eb-48a4a3d5417c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:36.283923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ab264-bc82-4041-9add-2ea4bae38d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:36.284969 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ab264-bc82-4041-9add-2ea4bae38d33 None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:36.285358 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ab264-bc82-4041-9add-2ea4bae38d33 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:36.285734 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ab264-bc82-4041-9add-2ea4bae38d33 None admin] PATH_INFO: `/v3/projects/0500e378ea374ca692e230bbd8781318/users/02fcd7cc86134ce488c7dd75dfb174b8/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:36.287885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ab264-bc82-4041-9add-2ea4bae38d33 None admin] RBAC: Authorizing `identity:create_grant(project_id=0500e378ea374ca692e230bbd8781318, user_id=02fcd7cc86134ce488c7dd75dfb174b8, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:36.289415 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ab264-bc82-4041-9add-2ea4bae38d33 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:36.305343 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 375/746] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:49:36 2022] PUT /identity/v3/projects/0500e378ea374ca692e230bbd8781318/users/02fcd7cc86134ce488c7dd75dfb174b8/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:49:36.306376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ab264-bc82-4041-9add-2ea4bae38d33 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:36.425026 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ab264-bc82-4041-9add-2ea4bae38d33 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:36.451951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56ca5b-330b-49a2-b8eb-48a4a3d5417c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:36.465294 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995c6b4-9b85-4869-8396-43dcfc57d99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:36.466143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995c6b4-9b85-4869-8396-43dcfc57d99e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:36.466422 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995c6b4-9b85-4869-8396-43dcfc57d99e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:36.466855 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995c6b4-9b85-4869-8396-43dcfc57d99e None admin] PATH_INFO: `/v3/users/swift` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:36.476543 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995c6b4-9b85-4869-8396-43dcfc57d99e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:36.478021 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995c6b4-9b85-4869-8396-43dcfc57d99e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:36.495992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-7995c6b4-9b85-4869-8396-43dcfc57d99e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 22:49:36.495992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:36.495992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:36.495992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:36.495992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:36.495992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:36.495992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:36.495992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:36.495992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:36.495992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:36.495992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:36.495992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:36.495992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:36.495992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:36.495992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:36.495992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:36.498213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:36.498213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:36.498213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:36.498213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:36.498213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:36.498213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:36.498213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:36.498213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:36.498213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:36.498213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:36.498213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:36.498213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:36.498213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:36.498213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:36.498213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:36.498213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:36.498213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:36.498213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:36.499251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:36.499251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:36.499251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:36.499251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:36.499251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:36.499251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:36.499251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:36.499251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:36.499251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 22:49:36.499251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:36.499251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 376/747] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:49:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:36.506251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995c6b4-9b85-4869-8396-43dcfc57d99e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:36.524413 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84efbf-f031-4d8b-9ad9-c54625d9c3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:36.525224 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84efbf-f031-4d8b-9ad9-c54625d9c3e2 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:36.525502 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84efbf-f031-4d8b-9ad9-c54625d9c3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:36.525773 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84efbf-f031-4d8b-9ad9-c54625d9c3e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:36.547612 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84efbf-f031-4d8b-9ad9-c54625d9c3e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:36.568798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-ae84efbf-f031-4d8b-9ad9-c54625d9c3e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:36.568798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:36.568798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:36.568798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:36.568798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:36.568798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:36.568798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:36.568798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:36.568798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:36.568798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:36.568798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:36.568798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:36.568798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:36.568798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:36.568798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:36.568798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:36.571121 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:36.571121 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:36.571121 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:36.571121 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:36.571121 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:36.571121 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:36.571121 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:36.571121 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:36.571121 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:36.571121 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:36.571121 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:36.571121 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:36.571121 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:36.571121 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:36.571121 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:36.571121 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:36.573179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:36.573179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:36.573179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:36.573179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:36.573179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:36.573179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:36.573179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:36.573179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:36.573179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:36.573179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:36.573179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:36.573179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:36.573179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 372/748] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 9 22:49:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:36.579180 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84efbf-f031-4d8b-9ad9-c54625d9c3e2 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:36.610906 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995c6b4-9b85-4869-8396-43dcfc57d99e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:36.652471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70e0cb-fd1e-4bb0-a49a-922d0bbcf658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:36.653154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70e0cb-fd1e-4bb0-a49a-922d0bbcf658 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:36.653436 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70e0cb-fd1e-4bb0-a49a-922d0bbcf658 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:36.653710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70e0cb-fd1e-4bb0-a49a-922d0bbcf658 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:36.654590 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f70e0cb-fd1e-4bb0-a49a-922d0bbcf658 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:36.654896 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f70e0cb-fd1e-4bb0-a49a-922d0bbcf658 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:36.656339 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f70e0cb-fd1e-4bb0-a49a-922d0bbcf658 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:36.674738 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 377/749] 10.222.0.23 () {56 vars in 1135 bytes} [Mon May 9 22:49:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:36.681040 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70e0cb-fd1e-4bb0-a49a-922d0bbcf658 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:36.739535 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84efbf-f031-4d8b-9ad9-c54625d9c3e2 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:36.773718 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06540b99-c05c-463a-8266-1fd32d0d70a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:36.774437 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06540b99-c05c-463a-8266-1fd32d0d70a4 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:36.774720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06540b99-c05c-463a-8266-1fd32d0d70a4 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:36.774999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06540b99-c05c-463a-8266-1fd32d0d70a4 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:36.775774 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06540b99-c05c-463a-8266-1fd32d0d70a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:36.776086 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06540b99-c05c-463a-8266-1fd32d0d70a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:36.777568 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06540b99-c05c-463a-8266-1fd32d0d70a4 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:36.787016 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 373/750] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 9 22:49:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:36.793060 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06540b99-c05c-463a-8266-1fd32d0d70a4 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:36.805429 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70e0cb-fd1e-4bb0-a49a-922d0bbcf658 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:36.878841 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8cef0-3193-4502-ad8f-1f2c50b17a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:36.879513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8cef0-3193-4502-ad8f-1f2c50b17a51 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:36.880368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8cef0-3193-4502-ad8f-1f2c50b17a51 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:36.880644 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8cef0-3193-4502-ad8f-1f2c50b17a51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:36.913895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06540b99-c05c-463a-8266-1fd32d0d70a4 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:36.953907 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca87b19-d295-43c7-b285-5bcef3749c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:36.954654 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca87b19-d295-43c7-b285-5bcef3749c8a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:36.954940 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca87b19-d295-43c7-b285-5bcef3749c8a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:36.955212 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca87b19-d295-43c7-b285-5bcef3749c8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:37.071061 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb8cef0-3193-4502-ad8f-1f2c50b17a51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:37.072539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb8cef0-3193-4502-ad8f-1f2c50b17a51 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:37.101213 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca87b19-d295-43c7-b285-5bcef3749c8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:37.102756 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca87b19-d295-43c7-b285-5bcef3749c8a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:37.122712 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-7ca87b19-d295-43c7-b285-5bcef3749c8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:37.122712 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:37.122712 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:37.122712 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:37.122712 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:37.122712 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:37.122712 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:37.122712 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:37.122712 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:37.122712 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:37.122712 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:37.122712 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:37.122712 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:37.122712 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:37.122712 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:37.123682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:37.123682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:37.123682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:37.123682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:37.123682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:37.123682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:37.123682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:37.123682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:37.123682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:37.123682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:37.123682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:37.123682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:37.123682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:37.123682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:37.123682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:37.123682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:37.123682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:37.124631 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:37.124631 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:37.124631 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:37.124631 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:37.124631 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 374/751] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:37.132191 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca87b19-d295-43c7-b285-5bcef3749c8a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:37.222067 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 378/752] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:49:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 22:49:37.224085 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd9688-1dfc-4cb9-bb69-d4bf166babd6 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:37.224374 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd9688-1dfc-4cb9-bb69-d4bf166babd6 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:37.224648 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd9688-1dfc-4cb9-bb69-d4bf166babd6 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:37.230067 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 379/753] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:37.230908 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd9688-1dfc-4cb9-bb69-d4bf166babd6 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:37.240669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca87b19-d295-43c7-b285-5bcef3749c8a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:37.278410 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8cb49-a6ab-4913-8752-a79974f8cf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:37.279074 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8cb49-a6ab-4913-8752-a79974f8cf2c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:37.279346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8cb49-a6ab-4913-8752-a79974f8cf2c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:37.279613 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8cb49-a6ab-4913-8752-a79974f8cf2c None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:37.280323 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8cb49-a6ab-4913-8752-a79974f8cf2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:37.280619 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8cb49-a6ab-4913-8752-a79974f8cf2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:37.281975 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8cb49-a6ab-4913-8752-a79974f8cf2c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:37.298423 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 375/754] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:37.300030 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a085627-3d2b-4a83-8a97-5583a89a7e16 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:37.300309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a085627-3d2b-4a83-8a97-5583a89a7e16 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:37.300585 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a085627-3d2b-4a83-8a97-5583a89a7e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:37.318202 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-1a085627-3d2b-4a83-8a97-5583a89a7e16 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:37.319367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1a085627-3d2b-4a83-8a97-5583a89a7e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:37.339133 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd9688-1dfc-4cb9-bb69-d4bf166babd6 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:37.382357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7993f3-25f9-49da-9ce2-ebaf77a00ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:37.383474 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7993f3-25f9-49da-9ce2-ebaf77a00ca6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:37.383755 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7993f3-25f9-49da-9ce2-ebaf77a00ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:37.384027 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7993f3-25f9-49da-9ce2-ebaf77a00ca6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:37.397690 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7993f3-25f9-49da-9ce2-ebaf77a00ca6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:37.399816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7993f3-25f9-49da-9ce2-ebaf77a00ca6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:37.410204 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 376/755] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:49:37.413225 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a085627-3d2b-4a83-8a97-5583a89a7e16 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:37.431130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-5b7993f3-25f9-49da-9ce2-ebaf77a00ca6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:37.431130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:37.431130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:37.431130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:37.431130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:37.431130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:37.431130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:37.431130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:37.431130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:37.431130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:37.431130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:37.431130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:37.431130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:37.431130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:37.431130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:37.433047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:37.433047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:37.433047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:37.433047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:37.433047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:37.433047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:37.433047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:37.433047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:37.433047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:37.433047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:37.433047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:37.433047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:37.433047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:37.433047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:37.433047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:37.433047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:37.433047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:37.435247 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:37.435247 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:37.435247 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:37.435247 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:37.435247 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 380/756] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:37.435247 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7993f3-25f9-49da-9ce2-ebaf77a00ca6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:37.536789 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a085627-3d2b-4a83-8a97-5583a89a7e16 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:37.546941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7993f3-25f9-49da-9ce2-ebaf77a00ca6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:37.574937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2ac65-049e-49e9-ac69-34a9e3572938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:37.576011 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2ac65-049e-49e9-ac69-34a9e3572938 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:37.576286 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2ac65-049e-49e9-ac69-34a9e3572938 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:37.576554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2ac65-049e-49e9-ac69-34a9e3572938 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:37.582175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e608e6-3429-43e7-83fa-1bd8c03dcbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:37.583061 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e608e6-3429-43e7-83fa-1bd8c03dcbdc None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:37.583339 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e608e6-3429-43e7-83fa-1bd8c03dcbdc None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:37.583610 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e608e6-3429-43e7-83fa-1bd8c03dcbdc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:37.584299 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e608e6-3429-43e7-83fa-1bd8c03dcbdc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:37.584475 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2ac65-049e-49e9-ac69-34a9e3572938 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:37.585659 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e608e6-3429-43e7-83fa-1bd8c03dcbdc None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:37.585839 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2ac65-049e-49e9-ac69-34a9e3572938 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:37.600221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-d0e608e6-3429-43e7-83fa-1bd8c03dcbdc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 22:49:37.600221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:37.600221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:49:37.600221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:49:37.600221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:49:37.600221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:49:37.600221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:49:37.600221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:37.600221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 22:49:37.600221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:37.600221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:49:37.600221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:37.600221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:37.600221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:37.600221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:37.600221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:37.602437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:37.602437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:37.602437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:37.602437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:37.602437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:37.602437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:37.602437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:37.602437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:49:37.602437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:49:37.602437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:49:37.602437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:49:37.602437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:37.602437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:37.602437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:37.602437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:37.602437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:37.604464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:37.604464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:37.604464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:37.604464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:37.604464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:37.604464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:37.604464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:37.604464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:37.604464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:37.604464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:37.604464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:37.604464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:49:37.604464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:37.604464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 22:49:37.604464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:37.606384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-cbd2ac65-049e-49e9-ac69-34a9e3572938 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 22:49:37.606384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:37.606384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:37.606384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:37.606384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:37.606384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:37.606384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:37.606384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:37.606384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:37.606384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:37.606384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:37.606384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:37.606384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:37.606384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:37.606384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:37.606384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:37.607810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:37.607810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:37.607810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:37.607810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:37.607810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:37.607810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:37.607810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:37.607810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:37.607810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:37.607810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:37.607810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:37.607810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:37.607810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:37.607810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:37.607810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:37.607810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:37.607810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:37.607810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:37.608962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:37.608962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:37.608962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:37.608962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:37.608962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:37.608962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:37.608962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:37.608962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:37.608962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 22:49:37.608962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:37.609896 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 381/757] 10.222.0.23 () {56 vars in 1129 bytes} [Mon May 9 22:49:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:37.609896 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e608e6-3429-43e7-83fa-1bd8c03dcbdc None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:37.622174 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 377/758] 10.222.0.23 () {56 vars in 1117 bytes} [Mon May 9 22:49:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:37.622174 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2ac65-049e-49e9-ac69-34a9e3572938 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:37.729580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e608e6-3429-43e7-83fa-1bd8c03dcbdc None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:37.731414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2ac65-049e-49e9-ac69-34a9e3572938 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:37.763336 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f751a1b-ae18-40f7-94d3-68992f8e0943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:37.764269 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f751a1b-ae18-40f7-94d3-68992f8e0943 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:37.764552 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f751a1b-ae18-40f7-94d3-68992f8e0943 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:37.765000 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f751a1b-ae18-40f7-94d3-68992f8e0943 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:37.766189 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f751a1b-ae18-40f7-94d3-68992f8e0943 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:37.766507 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f751a1b-ae18-40f7-94d3-68992f8e0943 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:37.768260 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f751a1b-ae18-40f7-94d3-68992f8e0943 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:37.771455 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489a18f-7137-4135-a19e-6d328286585c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:37.772190 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489a18f-7137-4135-a19e-6d328286585c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:37.772532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489a18f-7137-4135-a19e-6d328286585c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:37.772873 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489a18f-7137-4135-a19e-6d328286585c None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:37.773636 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b489a18f-7137-4135-a19e-6d328286585c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:37.774005 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b489a18f-7137-4135-a19e-6d328286585c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:37.775459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b489a18f-7137-4135-a19e-6d328286585c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:37.784964 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 382/759] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:37.786658 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 378/760] 10.222.0.23 () {56 vars in 1124 bytes} [Mon May 9 22:49:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:37.786977 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f751a1b-ae18-40f7-94d3-68992f8e0943 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:37.787747 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489a18f-7137-4135-a19e-6d328286585c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:37.905709 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489a18f-7137-4135-a19e-6d328286585c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:37.908082 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f751a1b-ae18-40f7-94d3-68992f8e0943 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:37.943997 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5889f7-8fec-4875-8393-579d29f7c339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:37.944666 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5889f7-8fec-4875-8393-579d29f7c339 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:37.944936 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5889f7-8fec-4875-8393-579d29f7c339 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:37.945209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5889f7-8fec-4875-8393-579d29f7c339 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:37.945907 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5889f7-8fec-4875-8393-579d29f7c339 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:37.946232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5889f7-8fec-4875-8393-579d29f7c339 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:37.947685 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5889f7-8fec-4875-8393-579d29f7c339 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:37.950842 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef6544-b668-42af-a623-820384184cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:37.951567 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef6544-b668-42af-a623-820384184cc4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:37.951851 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef6544-b668-42af-a623-820384184cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:37.952124 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef6544-b668-42af-a623-820384184cc4 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:37.952946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caef6544-b668-42af-a623-820384184cc4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:37.953250 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caef6544-b668-42af-a623-820384184cc4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:37.953431 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 379/761] 10.222.0.23 () {56 vars in 1124 bytes} [Mon May 9 22:49:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:37.954251 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5889f7-8fec-4875-8393-579d29f7c339 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:37.954861 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caef6544-b668-42af-a623-820384184cc4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:37.973993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 383/762] 10.222.0.23 () {56 vars in 1149 bytes} [Mon May 9 22:49:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:37.974911 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef6544-b668-42af-a623-820384184cc4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:38.066001 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5889f7-8fec-4875-8393-579d29f7c339 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:38.074242 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef6544-b668-42af-a623-820384184cc4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:38.100456 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca38ebe-87ef-459d-86b4-47d6c80e8963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:38.101231 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca38ebe-87ef-459d-86b4-47d6c80e8963 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:38.101500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca38ebe-87ef-459d-86b4-47d6c80e8963 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:38.101768 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca38ebe-87ef-459d-86b4-47d6c80e8963 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:38.108187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca38ebe-87ef-459d-86b4-47d6c80e8963 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:38.109412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03fed39-3c8c-4aac-93dc-5468c8d0ef6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:38.109864 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca38ebe-87ef-459d-86b4-47d6c80e8963 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:38.110502 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fed39-3c8c-4aac-93dc-5468c8d0ef6a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:38.110957 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fed39-3c8c-4aac-93dc-5468c8d0ef6a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:38.111397 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fed39-3c8c-4aac-93dc-5468c8d0ef6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:38.112341 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03fed39-3c8c-4aac-93dc-5468c8d0ef6a None admin] RBAC: Adding query filter params (interface=public, service_id=abfd02addad54183b9d9795a25f93eea, region_id=RegionOne) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:38.112825 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03fed39-3c8c-4aac-93dc-5468c8d0ef6a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:38.114418 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03fed39-3c8c-4aac-93dc-5468c8d0ef6a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:38.121950 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 384/763] 10.222.0.23 () {56 vars in 1251 bytes} [Mon May 9 22:49:37 2022] GET /identity/v3/endpoints?service_id=abfd02addad54183b9d9795a25f93eea&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:38.122920 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03fed39-3c8c-4aac-93dc-5468c8d0ef6a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:38.123472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-4ca38ebe-87ef-459d-86b4-47d6c80e8963 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:38.123472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:38.123472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:38.123472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:38.123472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:38.123472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:38.123472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:38.123472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:38.123472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:38.123472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:38.123472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:38.123472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:38.123472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:38.123472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:38.123472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:38.124371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:38.124371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:38.124371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:38.124371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:38.124371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:38.124371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:38.124371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:38.124371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:38.124371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:38.124371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:38.124371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:38.124371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:38.124371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:38.124371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:38.124371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:38.124371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:38.124371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:38.125298 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:38.125298 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:38.125298 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:38.125298 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:38.125298 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:38.125298 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:38.125298 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:38.125298 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:38.125298 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 380/764] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:38.126108 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15697a2-eb5e-465b-bd76-9bb1d3e225f4 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:38.126368 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15697a2-eb5e-465b-bd76-9bb1d3e225f4 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:38.126639 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15697a2-eb5e-465b-bd76-9bb1d3e225f4 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:38.130147 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 381/765] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:38.131248 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15697a2-eb5e-465b-bd76-9bb1d3e225f4 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:38.235248 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15697a2-eb5e-465b-bd76-9bb1d3e225f4 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:38.269676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92802b-944e-4433-965a-c6c7694c5dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:38.270912 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92802b-944e-4433-965a-c6c7694c5dc0 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:38.271189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92802b-944e-4433-965a-c6c7694c5dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:38.271458 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92802b-944e-4433-965a-c6c7694c5dc0 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:38.272237 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92802b-944e-4433-965a-c6c7694c5dc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:38.272544 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92802b-944e-4433-965a-c6c7694c5dc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:38.273978 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92802b-944e-4433-965a-c6c7694c5dc0 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:38.285490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 382/766] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:38.287180 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92802b-944e-4433-965a-c6c7694c5dc0 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:38.358315 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03fed39-3c8c-4aac-93dc-5468c8d0ef6a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:38.386784 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92802b-944e-4433-965a-c6c7694c5dc0 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:38.394812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa195e9-06f1-4f6b-a153-c8caa6f0d089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:38.396280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa195e9-06f1-4f6b-a153-c8caa6f0d089 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:38.396787 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa195e9-06f1-4f6b-a153-c8caa6f0d089 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:38.397300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa195e9-06f1-4f6b-a153-c8caa6f0d089 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:38.419143 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d81d32-f336-46ec-b67e-3e8c4e12f012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:38.420045 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d81d32-f336-46ec-b67e-3e8c4e12f012 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:38.420329 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d81d32-f336-46ec-b67e-3e8c4e12f012 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:38.420607 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d81d32-f336-46ec-b67e-3e8c4e12f012 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:38.421302 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d81d32-f336-46ec-b67e-3e8c4e12f012 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:38.422712 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d81d32-f336-46ec-b67e-3e8c4e12f012 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:38.427726 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 383/767] 10.222.0.23 () {56 vars in 1087 bytes} [Mon May 9 22:49:38 2022] GET /identity/v3/services => generated 772 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:38.428910 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b206d-d34b-4008-bead-127d9ba210bc None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:38.429196 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b206d-d34b-4008-bead-127d9ba210bc None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:38.429474 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b206d-d34b-4008-bead-127d9ba210bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:38.448301 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5d3b206d-d34b-4008-bead-127d9ba210bc None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:38.449598 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5d3b206d-d34b-4008-bead-127d9ba210bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:38.492477 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa195e9-06f1-4f6b-a153-c8caa6f0d089 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:38.494929 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa195e9-06f1-4f6b-a153-c8caa6f0d089 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:38.524081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 384/768] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:49:38.525097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b206d-d34b-4008-bead-127d9ba210bc None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:38.630573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b206d-d34b-4008-bead-127d9ba210bc None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:38.632534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 385/769] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:49:38 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 22:49:38.634512 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa195e9-06f1-4f6b-a153-c8caa6f0d089 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:38.674060 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db83a0-fb8d-4a6e-9855-ab751e551350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:38.674859 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db83a0-fb8d-4a6e-9855-ab751e551350 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:38.675141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db83a0-fb8d-4a6e-9855-ab751e551350 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:38.675412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db83a0-fb8d-4a6e-9855-ab751e551350 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:38.676240 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db83a0-fb8d-4a6e-9855-ab751e551350 None admin] RBAC: Adding query filter params (role.id=bda550492f4f4c5c83f52fa89fc18a7d, scope.project.id=0500e378ea374ca692e230bbd8781318, user.id=8cb6c294ce1842b68a6bc73ac381ed6b) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:38.677135 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db83a0-fb8d-4a6e-9855-ab751e551350 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:38.679161 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db83a0-fb8d-4a6e-9855-ab751e551350 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:38.686500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 385/770] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:49:38 2022] GET /identity/v3/role_assignments?user.id=8cb6c294ce1842b68a6bc73ac381ed6b&scope.project.id=0500e378ea374ca692e230bbd8781318&role.id=bda550492f4f4c5c83f52fa89fc18a7d => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:38.688283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db83a0-fb8d-4a6e-9855-ab751e551350 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:38.811567 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa195e9-06f1-4f6b-a153-c8caa6f0d089 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:38.853659 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac53bcb-49af-4940-9cfc-88cfd56c6383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:38.856962 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac53bcb-49af-4940-9cfc-88cfd56c6383 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:38.858639 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac53bcb-49af-4940-9cfc-88cfd56c6383 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:38.860081 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac53bcb-49af-4940-9cfc-88cfd56c6383 None admin] PATH_INFO: `/v3/services/network` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:38.861095 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac53bcb-49af-4940-9cfc-88cfd56c6383 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:38.864765 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac53bcb-49af-4940-9cfc-88cfd56c6383 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:38.883964 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-0ac53bcb-49af-4940-9cfc-88cfd56c6383 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 09 22:49:38.883964 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:38.883964 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:49:38.883964 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:49:38.883964 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:49:38.883964 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:49:38.883964 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:49:38.883964 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:38.883964 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 22:49:38.883964 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:38.883964 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:49:38.883964 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:38.883964 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:38.883964 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:38.883964 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:38.883964 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:38.883964 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:38.885203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:38.885203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:38.885203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:38.885203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:38.885203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:38.885203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:38.885203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:49:38.885203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:49:38.885203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:49:38.885203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:49:38.885203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:38.885203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:38.885203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:38.885203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:38.885203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:38.885203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:38.886255 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:38.886255 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:38.886255 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:38.886255 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:38.886255 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:38.886255 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:38.886255 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:38.886255 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:38.886255 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:38.886255 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:38.886255 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:49:38.886255 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:38.886255 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 22:49:38.886255 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:38.900384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db83a0-fb8d-4a6e-9855-ab751e551350 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:38.900895 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 386/771] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:38 2022] GET /identity/v3/services/network => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:38.901942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac53bcb-49af-4940-9cfc-88cfd56c6383 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:38.969550 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829997b-08c7-4d1f-b463-8bc84242e2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:38.971661 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829997b-08c7-4d1f-b463-8bc84242e2e1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:38.972648 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829997b-08c7-4d1f-b463-8bc84242e2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:38.973277 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829997b-08c7-4d1f-b463-8bc84242e2e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:38.996252 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6829997b-08c7-4d1f-b463-8bc84242e2e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:38.997604 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6829997b-08c7-4d1f-b463-8bc84242e2e1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:39.017677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-6829997b-08c7-4d1f-b463-8bc84242e2e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:39.017677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:39.017677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:39.017677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:39.017677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:39.017677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:39.017677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:39.017677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:39.017677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:39.017677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:39.017677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:39.017677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:39.017677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:39.017677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:39.017677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:39.018898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:39.018898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:39.018898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:39.018898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:39.018898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:39.018898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:39.018898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:39.018898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:39.018898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:39.018898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:39.018898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:39.018898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:39.018898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:39.018898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:39.018898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:39.018898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:39.018898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:39.020007 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:39.020007 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:39.020007 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:39.020007 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:39.024034 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 386/772] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:39.025411 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78879603-0897-4a22-bfa7-9f8f401ff5dc None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:39.025690 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78879603-0897-4a22-bfa7-9f8f401ff5dc None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:39.025965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78879603-0897-4a22-bfa7-9f8f401ff5dc None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:39.031890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 387/773] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:39.032713 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78879603-0897-4a22-bfa7-9f8f401ff5dc None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:39.060574 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac53bcb-49af-4940-9cfc-88cfd56c6383 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:39.096991 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da2709-27ce-4430-96cf-4bc1710fd945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:39.097828 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da2709-27ce-4430-96cf-4bc1710fd945 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:39.098232 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da2709-27ce-4430-96cf-4bc1710fd945 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:39.099164 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da2709-27ce-4430-96cf-4bc1710fd945 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:39.099970 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da2709-27ce-4430-96cf-4bc1710fd945 None admin] RBAC: Adding query filter params (name=network) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:39.100385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da2709-27ce-4430-96cf-4bc1710fd945 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:39.101839 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da2709-27ce-4430-96cf-4bc1710fd945 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:39.106785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 387/774] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 9 22:49:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:39.108166 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a292b-c71e-4908-845f-e19a3e7aa9cb None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:39.108555 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a292b-c71e-4908-845f-e19a3e7aa9cb None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:39.108934 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a292b-c71e-4908-845f-e19a3e7aa9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:39.126641 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b58a292b-c71e-4908-845f-e19a3e7aa9cb None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:39.127966 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b58a292b-c71e-4908-845f-e19a3e7aa9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:39.161309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78879603-0897-4a22-bfa7-9f8f401ff5dc None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:39.195171 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 388/775] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:49:39.196212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a292b-c71e-4908-845f-e19a3e7aa9cb None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:39.202394 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c215b72-711a-43a9-9b13-8749d9551db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:39.203109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c215b72-711a-43a9-9b13-8749d9551db0 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:39.203392 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c215b72-711a-43a9-9b13-8749d9551db0 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:39.203724 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c215b72-711a-43a9-9b13-8749d9551db0 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:39.204567 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c215b72-711a-43a9-9b13-8749d9551db0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:39.204870 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c215b72-711a-43a9-9b13-8749d9551db0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:39.206321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c215b72-711a-43a9-9b13-8749d9551db0 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:39.228983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 388/776] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:39.234865 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c215b72-711a-43a9-9b13-8749d9551db0 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:39.298624 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a292b-c71e-4908-845f-e19a3e7aa9cb None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:39.332811 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44622bba-40db-4d7d-bda1-d04805322e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:39.333484 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44622bba-40db-4d7d-bda1-d04805322e75 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:39.333765 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44622bba-40db-4d7d-bda1-d04805322e75 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:39.334036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44622bba-40db-4d7d-bda1-d04805322e75 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:39.334775 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44622bba-40db-4d7d-bda1-d04805322e75 None admin] RBAC: Adding query filter params (type=network) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:39.335080 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44622bba-40db-4d7d-bda1-d04805322e75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:39.336448 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44622bba-40db-4d7d-bda1-d04805322e75 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:39.337874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c215b72-711a-43a9-9b13-8749d9551db0 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:39.341984 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 389/777] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 9 22:49:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:39.342982 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44622bba-40db-4d7d-bda1-d04805322e75 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:39.377992 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97587a9d-e81e-4723-9486-8f92d8bb13bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:39.379618 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97587a9d-e81e-4723-9486-8f92d8bb13bc None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:39.379901 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97587a9d-e81e-4723-9486-8f92d8bb13bc None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:39.380172 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97587a9d-e81e-4723-9486-8f92d8bb13bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:39.389063 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97587a9d-e81e-4723-9486-8f92d8bb13bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:39.391128 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97587a9d-e81e-4723-9486-8f92d8bb13bc None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:39.409824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-97587a9d-e81e-4723-9486-8f92d8bb13bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:39.409824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:39.409824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:39.409824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:39.409824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:39.409824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:39.409824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:39.409824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:39.409824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:39.409824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:39.409824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:39.409824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:39.409824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:39.409824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:39.409824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:39.412369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:39.412369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:39.412369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:39.412369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:39.412369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:39.412369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:39.412369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:39.412369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:39.412369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:39.412369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:39.412369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:39.412369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:39.412369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:39.412369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:39.412369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:39.412369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:39.412369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:39.413838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:39.413838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:39.413838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:39.413838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:39.413838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:39.413838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:39.413838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:39.413838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:39.422141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 389/778] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:39.422141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97587a9d-e81e-4723-9486-8f92d8bb13bc None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:39.446571 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44622bba-40db-4d7d-bda1-d04805322e75 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:39.487401 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845c1be-49a2-486d-8550-a6c081414fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:39.488369 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845c1be-49a2-486d-8550-a6c081414fa2 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:39.488748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845c1be-49a2-486d-8550-a6c081414fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:39.489648 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845c1be-49a2-486d-8550-a6c081414fa2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:39.499909 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845c1be-49a2-486d-8550-a6c081414fa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:39.508252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-0845c1be-49a2-486d-8550-a6c081414fa2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:49:39.508252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:39.508252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:39.508252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:39.508252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:39.508252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:39.508252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:39.508252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:39.508252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:39.508252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:39.508252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:39.508252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:39.508252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:39.508252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:39.508252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:39.508252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:39.510101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:39.510101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:39.510101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:39.510101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:39.510101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:39.510101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:39.510101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:39.510101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:39.510101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:39.510101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:39.510101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:39.510101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:39.510101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:39.510101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:39.510101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:39.510101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:39.512158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:39.512158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:39.512158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:39.512158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:39.512158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:39.512158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:39.512158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:39.512158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:39.512158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:39.512158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:39.512158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:49:39.512158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:39.512158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 390/779] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:49:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:39.512158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845c1be-49a2-486d-8550-a6c081414fa2 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:39.539969 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97587a9d-e81e-4723-9486-8f92d8bb13bc None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:39.586062 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665acb1-f136-4b6a-8de0-9bbd29ef172e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:39.587264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665acb1-f136-4b6a-8de0-9bbd29ef172e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:39.587560 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665acb1-f136-4b6a-8de0-9bbd29ef172e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:39.587838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665acb1-f136-4b6a-8de0-9bbd29ef172e None admin] PATH_INFO: `/v3/endpoints` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:39.588633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4665acb1-f136-4b6a-8de0-9bbd29ef172e None admin] RBAC: Adding query filter params (interface=public, service_id=08fa1a1d43e74ad6b38619335b484530, region_id=RegionOne) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:39.589294 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4665acb1-f136-4b6a-8de0-9bbd29ef172e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:39.591035 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4665acb1-f136-4b6a-8de0-9bbd29ef172e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:39.601245 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 390/780] 10.222.0.23 () {56 vars in 1251 bytes} [Mon May 9 22:49:39 2022] GET /identity/v3/endpoints?service_id=08fa1a1d43e74ad6b38619335b484530&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:39.602283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665acb1-f136-4b6a-8de0-9bbd29ef172e None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:39.615728 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845c1be-49a2-486d-8550-a6c081414fa2 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:39.664610 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff98d17a-440d-4277-834e-1e2f669e0472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:39.665897 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98d17a-440d-4277-834e-1e2f669e0472 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:39.666425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98d17a-440d-4277-834e-1e2f669e0472 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:39.667036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98d17a-440d-4277-834e-1e2f669e0472 None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:39.668619 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff98d17a-440d-4277-834e-1e2f669e0472 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:39.669139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff98d17a-440d-4277-834e-1e2f669e0472 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:39.671545 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff98d17a-440d-4277-834e-1e2f669e0472 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:39.687082 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 391/781] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:39.688778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff98d17a-440d-4277-834e-1e2f669e0472 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:39.716840 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665acb1-f136-4b6a-8de0-9bbd29ef172e None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:39.763141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b783f7f-6743-4407-b9ec-434d58f38ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:39.764686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b783f7f-6743-4407-b9ec-434d58f38ce4 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:39.765311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b783f7f-6743-4407-b9ec-434d58f38ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:39.765881 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b783f7f-6743-4407-b9ec-434d58f38ce4 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:39.766998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b783f7f-6743-4407-b9ec-434d58f38ce4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:39.768278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b783f7f-6743-4407-b9ec-434d58f38ce4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:39.769837 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b783f7f-6743-4407-b9ec-434d58f38ce4 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:39.780317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 391/782] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:49:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:39.782037 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b783f7f-6743-4407-b9ec-434d58f38ce4 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:39.803887 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff98d17a-440d-4277-834e-1e2f669e0472 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:39.845996 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f026410-e232-46a1-8ecb-fb0cd8539ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:39.847103 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f026410-e232-46a1-8ecb-fb0cd8539ccd None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:39.847384 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f026410-e232-46a1-8ecb-fb0cd8539ccd None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:39.847660 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f026410-e232-46a1-8ecb-fb0cd8539ccd None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:39.849215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f026410-e232-46a1-8ecb-fb0cd8539ccd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:39.851155 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f026410-e232-46a1-8ecb-fb0cd8539ccd None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:39.858369 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 392/783] 10.222.0.23 () {56 vars in 1087 bytes} [Mon May 9 22:49:39 2022] GET /identity/v3/services => generated 772 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:39.859801 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f026410-e232-46a1-8ecb-fb0cd8539ccd None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:39.903327 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b783f7f-6743-4407-b9ec-434d58f38ce4 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:39.938804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c0834-3c3c-4536-997b-d132dc5ebd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:39.939830 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c0834-3c3c-4536-997b-d132dc5ebd12 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:39.940317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c0834-3c3c-4536-997b-d132dc5ebd12 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:39.940795 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c0834-3c3c-4536-997b-d132dc5ebd12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:39.941835 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c0834-3c3c-4536-997b-d132dc5ebd12 None admin] RBAC: Adding query filter params (role.id=44847d50e6cb4bc7938c59b4e2d99833, scope.project.id=0500e378ea374ca692e230bbd8781318, user.id=1a6f50dcebf146baae8a9f0b5b112795) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:39.942914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c0834-3c3c-4536-997b-d132dc5ebd12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:39.944663 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c0834-3c3c-4536-997b-d132dc5ebd12 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:39.955307 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 392/784] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:49:39 2022] GET /identity/v3/role_assignments?user.id=1a6f50dcebf146baae8a9f0b5b112795&scope.project.id=0500e378ea374ca692e230bbd8781318&role.id=44847d50e6cb4bc7938c59b4e2d99833 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:39.983182 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f026410-e232-46a1-8ecb-fb0cd8539ccd None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:40.031763 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17371caa-ef21-4c22-a37d-5d30251c0b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:40.032572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17371caa-ef21-4c22-a37d-5d30251c0b39 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:40.032845 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17371caa-ef21-4c22-a37d-5d30251c0b39 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:40.033112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17371caa-ef21-4c22-a37d-5d30251c0b39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:40.044633 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17371caa-ef21-4c22-a37d-5d30251c0b39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:40.046098 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17371caa-ef21-4c22-a37d-5d30251c0b39 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:40.065120 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-17371caa-ef21-4c22-a37d-5d30251c0b39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:40.065120 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:40.065120 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:40.065120 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:40.065120 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:40.065120 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:40.065120 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:40.065120 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:40.065120 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:40.065120 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:40.065120 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:40.065120 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:40.065120 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:40.065120 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:40.065120 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:40.066161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:40.066161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:40.066161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:40.066161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:40.066161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:40.066161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:40.066161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:40.066161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:40.066161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:40.066161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:40.066161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:40.066161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:40.066161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:40.066161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:40.066161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:40.066161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:40.066161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:40.067646 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:40.067646 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:40.067646 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:40.067646 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:40.067646 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 393/785] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:40.072832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c0834-3c3c-4536-997b-d132dc5ebd12 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:40.226397 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c0834-3c3c-4536-997b-d132dc5ebd12 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:40.264588 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde65b7-1af2-4030-a2a4-ad203d1e1c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:40.265250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde65b7-1af2-4030-a2a4-ad203d1e1c95 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:40.265523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde65b7-1af2-4030-a2a4-ad203d1e1c95 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:40.265790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde65b7-1af2-4030-a2a4-ad203d1e1c95 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:40.266532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde65b7-1af2-4030-a2a4-ad203d1e1c95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:40.266835 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde65b7-1af2-4030-a2a4-ad203d1e1c95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:40.268202 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde65b7-1af2-4030-a2a4-ad203d1e1c95 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:40.280458 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 393/786] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:40.285599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17371caa-ef21-4c22-a37d-5d30251c0b39 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:40.392643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17371caa-ef21-4c22-a37d-5d30251c0b39 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:40.430211 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb1e3e-40b8-4ea7-aea9-8312f888b7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:40.431008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb1e3e-40b8-4ea7-aea9-8312f888b7a7 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:40.431277 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb1e3e-40b8-4ea7-aea9-8312f888b7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:40.431546 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb1e3e-40b8-4ea7-aea9-8312f888b7a7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:40.446392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb1e3e-40b8-4ea7-aea9-8312f888b7a7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:40.447814 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb1e3e-40b8-4ea7-aea9-8312f888b7a7 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:40.468956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-48fb1e3e-40b8-4ea7-aea9-8312f888b7a7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 22:49:40.468956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:40.468956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:40.468956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:40.468956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:40.468956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:40.468956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:40.468956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:40.468956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:40.468956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:40.468956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:40.468956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:40.468956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:40.468956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:40.468956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:40.468956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:40.469993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:40.469993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:40.469993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:40.469993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:40.469993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:40.469993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:40.469993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:40.469993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:40.469993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:40.469993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:40.469993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:40.469993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:40.469993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:40.469993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:40.469993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:40.469993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:40.469993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:40.469993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:40.471009 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:40.471009 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:40.471009 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:40.471009 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:40.471009 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:40.471009 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:40.471009 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:40.471009 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:40.471009 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 22:49:40.471009 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:40.474208 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 394/787] 10.222.0.23 () {56 vars in 1093 bytes} [Mon May 9 22:49:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:40.475895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde65b7-1af2-4030-a2a4-ad203d1e1c95 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:40.632314 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde65b7-1af2-4030-a2a4-ad203d1e1c95 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:40.666418 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9466b3da-0794-43e6-b0ea-bf99e19d8ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:40.667099 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466b3da-0794-43e6-b0ea-bf99e19d8ff4 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:40.667382 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466b3da-0794-43e6-b0ea-bf99e19d8ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:40.667656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466b3da-0794-43e6-b0ea-bf99e19d8ff4 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:40.668497 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9466b3da-0794-43e6-b0ea-bf99e19d8ff4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:40.668803 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9466b3da-0794-43e6-b0ea-bf99e19d8ff4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:40.671909 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9466b3da-0794-43e6-b0ea-bf99e19d8ff4 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:40.690371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 394/788] 10.222.0.23 () {56 vars in 1133 bytes} [Mon May 9 22:49:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:40.695440 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb1e3e-40b8-4ea7-aea9-8312f888b7a7 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:40.827114 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb1e3e-40b8-4ea7-aea9-8312f888b7a7 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:40.865768 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2378027-cb5c-47b8-8087-d11e204e83b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:40.866437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2378027-cb5c-47b8-8087-d11e204e83b9 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:40.866715 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2378027-cb5c-47b8-8087-d11e204e83b9 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:40.866982 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2378027-cb5c-47b8-8087-d11e204e83b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:40.951271 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2378027-cb5c-47b8-8087-d11e204e83b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:40.952647 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2378027-cb5c-47b8-8087-d11e204e83b9 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:40.975848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725021ff-f50f-42eb-b511-4d74da8e288b None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:40.976178 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725021ff-f50f-42eb-b511-4d74da8e288b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:40.976411 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725021ff-f50f-42eb-b511-4d74da8e288b None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:40.978106 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 395/789] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:40.983703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe942e-b75f-4e9c-b6b5-8119a8a92bd1 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:40.983990 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe942e-b75f-4e9c-b6b5-8119a8a92bd1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:40.984525 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe942e-b75f-4e9c-b6b5-8119a8a92bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:41.043284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-08fe942e-b75f-4e9c-b6b5-8119a8a92bd1 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:41.044498 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-08fe942e-b75f-4e9c-b6b5-8119a8a92bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:41.063623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 395/790] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:49:40 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 22:49:41.069811 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2378027-cb5c-47b8-8087-d11e204e83b9 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:41.174465 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2378027-cb5c-47b8-8087-d11e204e83b9 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:41.207055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 396/791] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:49:41.209585 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc101d5-2109-4474-96a5-051e6b3ccfb7 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:41.209905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be2c9e-5dfd-4dd2-ac08-c3868d738a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:41.210076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc101d5-2109-4474-96a5-051e6b3ccfb7 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:41.210316 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc101d5-2109-4474-96a5-051e6b3ccfb7 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:41.211040 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be2c9e-5dfd-4dd2-ac08-c3868d738a27 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:41.211322 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be2c9e-5dfd-4dd2-ac08-c3868d738a27 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:41.211596 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be2c9e-5dfd-4dd2-ac08-c3868d738a27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:41.212413 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 397/792] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:41.219359 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32b66f-bff4-436e-afc3-92dd6b64c859 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:41.219910 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32b66f-bff4-436e-afc3-92dd6b64c859 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:41.220393 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32b66f-bff4-436e-afc3-92dd6b64c859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:41.224411 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be2c9e-5dfd-4dd2-ac08-c3868d738a27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:41.226181 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be2c9e-5dfd-4dd2-ac08-c3868d738a27 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:41.237748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-1e32b66f-bff4-436e-afc3-92dd6b64c859 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:41.239304 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1e32b66f-bff4-436e-afc3-92dd6b64c859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:41.246368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-88be2c9e-5dfd-4dd2-ac08-c3868d738a27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:41.246368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:41.246368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:41.246368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:41.246368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:41.246368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:41.246368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:41.246368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:41.246368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:41.246368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:41.246368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:41.246368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:41.246368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:41.246368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:41.246368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:41.247288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:41.247288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:41.247288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:41.247288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:41.247288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:41.247288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:41.247288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:41.247288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:41.247288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:41.247288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:41.247288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:41.247288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:41.247288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:41.247288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:41.247288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:41.247288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:41.247288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:41.248223 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:41.248223 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:41.248223 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:41.248223 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:41.248223 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 396/793] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:41.251000 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be2c9e-5dfd-4dd2-ac08-c3868d738a27 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:41.311115 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 398/794] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:49:41.319250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32b66f-bff4-436e-afc3-92dd6b64c859 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:41.368616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be2c9e-5dfd-4dd2-ac08-c3868d738a27 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:41.413476 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0bf259-78cd-45df-9d73-7e5f01c8fa21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:41.414766 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0bf259-78cd-45df-9d73-7e5f01c8fa21 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:41.415272 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0bf259-78cd-45df-9d73-7e5f01c8fa21 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:41.415749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0bf259-78cd-45df-9d73-7e5f01c8fa21 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:41.416601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0bf259-78cd-45df-9d73-7e5f01c8fa21 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:41.418169 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0bf259-78cd-45df-9d73-7e5f01c8fa21 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:41.430154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-eb0bf259-78cd-45df-9d73-7e5f01c8fa21 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 22:49:41.430154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:41.430154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:49:41.430154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:49:41.430154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:49:41.430154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:49:41.430154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:49:41.430154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:41.430154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 22:49:41.430154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:41.430154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:49:41.430154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:41.430154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:41.430154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:41.430154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:41.430154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:41.431112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:41.431112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:41.431112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:41.431112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:41.431112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:41.431112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:41.431112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:41.431112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:49:41.431112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:49:41.431112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:49:41.431112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:49:41.431112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:41.431112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:41.431112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:41.431112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:41.431112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:41.431988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:41.431988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:41.431988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:41.431988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:41.431988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:41.431988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:41.431988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:41.431988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:41.431988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:41.431988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:41.431988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:41.431988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:49:41.431988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:41.431988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 22:49:41.431988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:41.432904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32b66f-bff4-436e-afc3-92dd6b64c859 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:41.434863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 397/795] 10.222.0.23 () {56 vars in 1129 bytes} [Mon May 9 22:49:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:41.435843 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0bf259-78cd-45df-9d73-7e5f01c8fa21 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:41.476310 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a4561-363c-498a-97b1-8b735cc07756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:41.476868 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a4561-363c-498a-97b1-8b735cc07756 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:41.477144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a4561-363c-498a-97b1-8b735cc07756 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:41.477430 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a4561-363c-498a-97b1-8b735cc07756 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:41.478154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a4561-363c-498a-97b1-8b735cc07756 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:41.478475 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a4561-363c-498a-97b1-8b735cc07756 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:41.480299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a4561-363c-498a-97b1-8b735cc07756 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:41.498197 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 399/796] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:41.499382 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a4561-363c-498a-97b1-8b735cc07756 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:41.552832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0bf259-78cd-45df-9d73-7e5f01c8fa21 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:41.599509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bec847-d08f-4735-8574-676c0cbf0c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:41.600328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bec847-d08f-4735-8574-676c0cbf0c96 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:41.600601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bec847-d08f-4735-8574-676c0cbf0c96 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:41.600925 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bec847-d08f-4735-8574-676c0cbf0c96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:41.614324 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bec847-d08f-4735-8574-676c0cbf0c96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:41.614851 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a4561-363c-498a-97b1-8b735cc07756 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:41.623413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-b7bec847-d08f-4735-8574-676c0cbf0c96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:49:41.623413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:41.623413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:41.623413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:41.623413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:41.623413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:41.623413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:41.623413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:41.623413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:41.623413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:41.623413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:41.623413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:41.623413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:41.623413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:41.623413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:41.623413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:41.624678 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:41.624678 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:41.624678 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:41.624678 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:41.624678 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:41.624678 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:41.624678 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:41.624678 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:41.624678 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:41.624678 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:41.624678 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:41.624678 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:41.624678 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:41.624678 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:41.624678 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:41.624678 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:41.625587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:41.625587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:41.625587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:41.625587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:41.625587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:41.625587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:41.625587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:41.625587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:41.625587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:41.625587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:41.625587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:49:41.625587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:41.630659 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 398/797] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:49:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:41.631451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bec847-d08f-4735-8574-676c0cbf0c96 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:41.657324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa4944a-2c86-4552-9c86-a97944870614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:41.658375 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa4944a-2c86-4552-9c86-a97944870614 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:41.658923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa4944a-2c86-4552-9c86-a97944870614 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:41.659441 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa4944a-2c86-4552-9c86-a97944870614 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:41.660349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa4944a-2c86-4552-9c86-a97944870614 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:41.660864 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa4944a-2c86-4552-9c86-a97944870614 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:41.662514 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa4944a-2c86-4552-9c86-a97944870614 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:41.668056 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 400/798] 10.222.0.23 () {56 vars in 1124 bytes} [Mon May 9 22:49:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:41.669103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa4944a-2c86-4552-9c86-a97944870614 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:41.750628 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bec847-d08f-4735-8574-676c0cbf0c96 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:41.770408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa4944a-2c86-4552-9c86-a97944870614 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:41.791429 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926d555-252f-4703-8055-dfd1f43c0d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:41.792215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926d555-252f-4703-8055-dfd1f43c0d5e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:41.792490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926d555-252f-4703-8055-dfd1f43c0d5e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:41.792758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926d555-252f-4703-8055-dfd1f43c0d5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:41.803329 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7926d555-252f-4703-8055-dfd1f43c0d5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:41.804087 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7926d555-252f-4703-8055-dfd1f43c0d5e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:41.807922 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb872b24-99d5-476c-8c54-e40f3056a51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:41.809876 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb872b24-99d5-476c-8c54-e40f3056a51c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:41.810403 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb872b24-99d5-476c-8c54-e40f3056a51c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:41.810884 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb872b24-99d5-476c-8c54-e40f3056a51c None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:41.811852 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb872b24-99d5-476c-8c54-e40f3056a51c None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:41.812398 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb872b24-99d5-476c-8c54-e40f3056a51c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:41.813952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb872b24-99d5-476c-8c54-e40f3056a51c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:41.818657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-7926d555-252f-4703-8055-dfd1f43c0d5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:41.818657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:41.818657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:41.818657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:41.818657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:41.818657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:41.818657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:41.818657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:41.818657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:41.818657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:41.818657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:41.818657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:41.818657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:41.818657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:41.818657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:41.819743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:41.819743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:41.819743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:41.819743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:41.819743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:41.819743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:41.819743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:41.819743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:41.819743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:41.819743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:41.819743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:41.819743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:41.819743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:41.819743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:41.819743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:41.819743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:41.819743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:41.820658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:41.820658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:41.820658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:41.820658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:41.820658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:41.820658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:41.820658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:41.820658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:41.824922 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 399/799] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:41.825750 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926d555-252f-4703-8055-dfd1f43c0d5e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:41.828925 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 401/800] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:49:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:41.829955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb872b24-99d5-476c-8c54-e40f3056a51c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:41.939044 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926d555-252f-4703-8055-dfd1f43c0d5e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:41.949204 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb872b24-99d5-476c-8c54-e40f3056a51c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:41.973288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b4ea1-9372-44d4-a9d2-6fb1c4c199cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:41.973953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b4ea1-9372-44d4-a9d2-6fb1c4c199cc None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:41.974277 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b4ea1-9372-44d4-a9d2-6fb1c4c199cc None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:41.974549 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b4ea1-9372-44d4-a9d2-6fb1c4c199cc None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:41.975315 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b4ea1-9372-44d4-a9d2-6fb1c4c199cc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:41.975616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b4ea1-9372-44d4-a9d2-6fb1c4c199cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:41.977025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b4ea1-9372-44d4-a9d2-6fb1c4c199cc None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:41.982079 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 400/801] 10.222.0.23 () {56 vars in 1124 bytes} [Mon May 9 22:49:41 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:41.983005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b4ea1-9372-44d4-a9d2-6fb1c4c199cc None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:41.992590 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eddd33-f9a7-4cc9-a488-e2c26f393096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:41.993554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eddd33-f9a7-4cc9-a488-e2c26f393096 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:41.994424 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eddd33-f9a7-4cc9-a488-e2c26f393096 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:41.994900 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eddd33-f9a7-4cc9-a488-e2c26f393096 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:41.996354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eddd33-f9a7-4cc9-a488-e2c26f393096 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:41.996945 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eddd33-f9a7-4cc9-a488-e2c26f393096 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:41.999115 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eddd33-f9a7-4cc9-a488-e2c26f393096 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:42.018421 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 402/802] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:42.019965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eddd33-f9a7-4cc9-a488-e2c26f393096 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:42.093126 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b4ea1-9372-44d4-a9d2-6fb1c4c199cc None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:42.131792 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383715e-2483-4df8-941e-3e3ddca7777e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:42.132632 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383715e-2483-4df8-941e-3e3ddca7777e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:42.132912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383715e-2483-4df8-941e-3e3ddca7777e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:42.133239 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383715e-2483-4df8-941e-3e3ddca7777e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:42.140572 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eddd33-f9a7-4cc9-a488-e2c26f393096 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:42.143835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383715e-2483-4df8-941e-3e3ddca7777e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:42.145521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383715e-2483-4df8-941e-3e3ddca7777e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:42.163832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-1383715e-2483-4df8-941e-3e3ddca7777e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:42.163832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:42.163832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:42.163832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:42.163832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:42.163832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:42.163832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:42.163832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:42.163832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:42.163832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:42.163832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:42.163832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:42.163832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:42.163832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:42.163832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:42.164774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:42.164774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:42.164774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:42.164774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:42.164774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:42.164774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:42.164774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:42.164774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:42.164774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:42.164774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:42.164774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:42.164774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:42.164774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:42.164774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:42.164774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:42.164774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:42.164774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:42.165665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:42.165665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:42.165665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:42.165665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:42.169766 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 401/803] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:42.170565 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383715e-2483-4df8-941e-3e3ddca7777e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:42.179254 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808141ed-e66d-4c5c-959d-add1d2115b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:42.180208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808141ed-e66d-4c5c-959d-add1d2115b55 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:42.180685 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808141ed-e66d-4c5c-959d-add1d2115b55 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:42.181152 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808141ed-e66d-4c5c-959d-add1d2115b55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:42.182086 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808141ed-e66d-4c5c-959d-add1d2115b55 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:42.183714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808141ed-e66d-4c5c-959d-add1d2115b55 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:42.207431 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 403/804] 10.222.0.23 () {60 vars in 1144 bytes} [Mon May 9 22:49:42 2022] POST /identity/v3/endpoints => generated 346 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:49:42.210027 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808141ed-e66d-4c5c-959d-add1d2115b55 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:42.294689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383715e-2483-4df8-941e-3e3ddca7777e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:42.336747 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37705b0-cf9b-498f-b631-e1ea2649e8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:42.337618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37705b0-cf9b-498f-b631-e1ea2649e8d9 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:42.338050 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37705b0-cf9b-498f-b631-e1ea2649e8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:42.338477 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37705b0-cf9b-498f-b631-e1ea2649e8d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:42.338732 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808141ed-e66d-4c5c-959d-add1d2115b55 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:42.339530 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37705b0-cf9b-498f-b631-e1ea2649e8d9 None admin] RBAC: Adding query filter params (role.id=bda550492f4f4c5c83f52fa89fc18a7d, scope.project.id=0500e378ea374ca692e230bbd8781318, user.id=02fcd7cc86134ce488c7dd75dfb174b8) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:42.339940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37705b0-cf9b-498f-b631-e1ea2649e8d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:42.342436 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37705b0-cf9b-498f-b631-e1ea2649e8d9 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:42.348819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 402/805] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:49:42 2022] GET /identity/v3/role_assignments?user.id=02fcd7cc86134ce488c7dd75dfb174b8&scope.project.id=0500e378ea374ca692e230bbd8781318&role.id=bda550492f4f4c5c83f52fa89fc18a7d => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:42.401630 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b5510-465a-4d77-a80e-3093ef388db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:42.402628 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b5510-465a-4d77-a80e-3093ef388db5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:42.403082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b5510-465a-4d77-a80e-3093ef388db5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:42.403501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b5510-465a-4d77-a80e-3093ef388db5 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:42.404422 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b5510-465a-4d77-a80e-3093ef388db5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:42.404884 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b5510-465a-4d77-a80e-3093ef388db5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:42.406499 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b5510-465a-4d77-a80e-3093ef388db5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:42.421718 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 404/806] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:42.428211 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37705b0-cf9b-498f-b631-e1ea2649e8d9 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:42.573691 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37705b0-cf9b-498f-b631-e1ea2649e8d9 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:42.611059 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51958576-df51-4ca3-9e4d-fd07f0e61f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:42.611879 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51958576-df51-4ca3-9e4d-fd07f0e61f6f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:42.612158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51958576-df51-4ca3-9e4d-fd07f0e61f6f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:42.612427 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51958576-df51-4ca3-9e4d-fd07f0e61f6f None admin] PATH_INFO: `/v3/users/swift` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:42.620786 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51958576-df51-4ca3-9e4d-fd07f0e61f6f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:42.627792 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51958576-df51-4ca3-9e4d-fd07f0e61f6f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:42.632539 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2cf18-9e2e-4cf5-9c57-abc12ed8ebea None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:42.632840 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2cf18-9e2e-4cf5-9c57-abc12ed8ebea None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:42.633083 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2cf18-9e2e-4cf5-9c57-abc12ed8ebea None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:42.634824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 405/807] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:42.641920 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe031cb-a479-4bd5-8e7c-0d3e1dd6cbcc None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:42.642220 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe031cb-a479-4bd5-8e7c-0d3e1dd6cbcc None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:42.642501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe031cb-a479-4bd5-8e7c-0d3e1dd6cbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:42.648376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-51958576-df51-4ca3-9e4d-fd07f0e61f6f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 22:49:42.648376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:42.648376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:42.648376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:42.648376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:42.648376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:42.648376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:42.648376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:42.648376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:42.648376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:42.648376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:42.648376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:42.648376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:42.648376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:42.648376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:42.648376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:42.649566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:42.649566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:42.649566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:42.649566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:42.649566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:42.649566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:42.649566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:42.649566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:42.649566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:42.649566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:42.649566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:42.649566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:42.649566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:42.649566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:42.649566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:42.649566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:42.649566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:42.649566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:42.650771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:42.650771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:42.650771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:42.650771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:42.650771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:42.650771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:42.650771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:42.650771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:42.650771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 22:49:42.650771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:42.653058 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 403/808] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:49:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:42.658298 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51958576-df51-4ca3-9e4d-fd07f0e61f6f None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:42.664966 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-ebe031cb-a479-4bd5-8e7c-0d3e1dd6cbcc None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:42.666314 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ebe031cb-a479-4bd5-8e7c-0d3e1dd6cbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:42.762139 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 406/809] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:49:42.774495 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51958576-df51-4ca3-9e4d-fd07f0e61f6f None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:42.781183 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe031cb-a479-4bd5-8e7c-0d3e1dd6cbcc None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:42.824497 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5b5e0-b586-4541-b506-c743ccbf1217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:42.825089 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5b5e0-b586-4541-b506-c743ccbf1217 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:42.825362 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5b5e0-b586-4541-b506-c743ccbf1217 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:42.825628 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5b5e0-b586-4541-b506-c743ccbf1217 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:42.826502 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec5b5e0-b586-4541-b506-c743ccbf1217 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:42.826792 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec5b5e0-b586-4541-b506-c743ccbf1217 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:42.828219 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec5b5e0-b586-4541-b506-c743ccbf1217 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:42.850486 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 404/810] 10.222.0.23 () {56 vars in 1135 bytes} [Mon May 9 22:49:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:42.851874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc0c37-1bb4-4e4c-8104-a848d4b64a1e None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:42.852159 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc0c37-1bb4-4e4c-8104-a848d4b64a1e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:42.852434 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc0c37-1bb4-4e4c-8104-a848d4b64a1e None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:42.854059 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 405/811] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:42.859031 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc0c37-1bb4-4e4c-8104-a848d4b64a1e None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:42.895122 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe031cb-a479-4bd5-8e7c-0d3e1dd6cbcc None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:42.938572 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae15757-e25c-4716-8ec9-ea288ef10722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:42.939701 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae15757-e25c-4716-8ec9-ea288ef10722 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:42.940158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae15757-e25c-4716-8ec9-ea288ef10722 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:42.941016 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae15757-e25c-4716-8ec9-ea288ef10722 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:42.954484 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae15757-e25c-4716-8ec9-ea288ef10722 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:42.965009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-7ae15757-e25c-4716-8ec9-ea288ef10722 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:42.965009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:42.965009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:42.965009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:42.965009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:42.965009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:42.965009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:42.965009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:42.965009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:42.965009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:42.965009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:42.965009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:42.965009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:42.965009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:42.965009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:42.965009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:42.965928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:42.965928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:42.965928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:42.965928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:42.965928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:42.965928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:42.965928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:42.965928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:42.965928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:42.965928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:42.965928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:42.965928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:42.965928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:42.965928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:42.965928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:42.965928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:42.966806 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:42.966806 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:42.966806 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:42.966806 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:42.966806 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:42.966806 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:42.966806 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:42.966806 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:42.966806 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:42.966806 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:42.966806 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:42.966806 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:42.967440 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc0c37-1bb4-4e4c-8104-a848d4b64a1e None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:42.970106 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 407/812] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 9 22:49:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:42.971536 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927c15a-0107-430c-a47f-e42c0dfb900b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:42.971949 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927c15a-0107-430c-a47f-e42c0dfb900b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:42.972370 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927c15a-0107-430c-a47f-e42c0dfb900b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:42.993173 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-8927c15a-0107-430c-a47f-e42c0dfb900b None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:42.994858 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-8927c15a-0107-430c-a47f-e42c0dfb900b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:43.011003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77ae2b-7d6d-47ba-8d49-bd403dcbbb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:43.012417 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77ae2b-7d6d-47ba-8d49-bd403dcbbb0b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:43.013110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77ae2b-7d6d-47ba-8d49-bd403dcbbb0b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:43.013528 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77ae2b-7d6d-47ba-8d49-bd403dcbbb0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:43.027045 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77ae2b-7d6d-47ba-8d49-bd403dcbbb0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:43.029435 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77ae2b-7d6d-47ba-8d49-bd403dcbbb0b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:43.052063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-9e77ae2b-7d6d-47ba-8d49-bd403dcbbb0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:43.052063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:43.052063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:43.052063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:43.052063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:43.052063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:43.052063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:43.052063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:43.052063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:43.052063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:43.052063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:43.052063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:43.052063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:43.052063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:43.052063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:43.053163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:43.053163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:43.053163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:43.053163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:43.053163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:43.053163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:43.053163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:43.053163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:43.053163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:43.053163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:43.053163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:43.053163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:43.053163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:43.053163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:43.053163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:43.053163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:43.053163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:43.054275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:43.054275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:43.054275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:43.054275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:43.056820 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 406/813] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:43.057930 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77ae2b-7d6d-47ba-8d49-bd403dcbbb0b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:43.069483 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 408/814] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:49:43.070436 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927c15a-0107-430c-a47f-e42c0dfb900b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:43.179328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77ae2b-7d6d-47ba-8d49-bd403dcbbb0b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:43.203029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927c15a-0107-430c-a47f-e42c0dfb900b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:43.212587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c8795-406f-4bd8-8e3c-ac80fd8c3013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:43.213284 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c8795-406f-4bd8-8e3c-ac80fd8c3013 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:43.213527 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c8795-406f-4bd8-8e3c-ac80fd8c3013 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:43.213798 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c8795-406f-4bd8-8e3c-ac80fd8c3013 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:43.214714 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68c8795-406f-4bd8-8e3c-ac80fd8c3013 None admin] RBAC: Adding query filter params (name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:43.215028 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68c8795-406f-4bd8-8e3c-ac80fd8c3013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:43.216437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68c8795-406f-4bd8-8e3c-ac80fd8c3013 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:43.225332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 407/815] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 9 22:49:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:43.226007 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c8795-406f-4bd8-8e3c-ac80fd8c3013 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:43.245376 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532912d8-7ed6-49af-92c7-51af20354acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:43.246874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532912d8-7ed6-49af-92c7-51af20354acd None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:43.247718 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532912d8-7ed6-49af-92c7-51af20354acd None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:43.248139 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532912d8-7ed6-49af-92c7-51af20354acd None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:43.249240 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532912d8-7ed6-49af-92c7-51af20354acd None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:43.249838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532912d8-7ed6-49af-92c7-51af20354acd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:43.251666 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532912d8-7ed6-49af-92c7-51af20354acd None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:43.267289 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 409/816] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:43.268340 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532912d8-7ed6-49af-92c7-51af20354acd None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:43.334903 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c8795-406f-4bd8-8e3c-ac80fd8c3013 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:43.370403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76a8a3-0e0c-4d4c-8781-2ffb757027b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:43.371188 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76a8a3-0e0c-4d4c-8781-2ffb757027b8 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:43.371650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76a8a3-0e0c-4d4c-8781-2ffb757027b8 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:43.372000 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76a8a3-0e0c-4d4c-8781-2ffb757027b8 None admin] PATH_INFO: `/v3/services/network` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:43.372659 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76a8a3-0e0c-4d4c-8781-2ffb757027b8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:43.374072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76a8a3-0e0c-4d4c-8781-2ffb757027b8 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:43.386595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-2b76a8a3-0e0c-4d4c-8781-2ffb757027b8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 09 22:49:43.386595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:43.386595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:49:43.386595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:49:43.386595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:49:43.386595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:49:43.386595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:49:43.386595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:43.386595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 22:49:43.386595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:43.386595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:49:43.386595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:43.386595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:43.386595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:43.386595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:43.386595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:43.386595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:43.387867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:43.387867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:43.387867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:43.387867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:43.387867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:43.387867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:43.387867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:49:43.387867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:49:43.387867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:49:43.387867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:49:43.387867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:43.387867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:43.387867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:43.387867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:43.387867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:43.387867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:43.388942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:43.388942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:43.388942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:43.388942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:43.388942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:43.388942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:43.388942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:43.388942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:43.388942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:43.388942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:43.388942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:49:43.388942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:43.388942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 22:49:43.388942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:43.393337 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 408/817] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:43 2022] GET /identity/v3/services/network => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:43.393905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532912d8-7ed6-49af-92c7-51af20354acd None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:43.394218 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76a8a3-0e0c-4d4c-8781-2ffb757027b8 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:43.429134 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9586ef8-3b83-4fb4-b808-eea08840b563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:43.430284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9586ef8-3b83-4fb4-b808-eea08840b563 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:43.430718 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9586ef8-3b83-4fb4-b808-eea08840b563 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:43.431158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9586ef8-3b83-4fb4-b808-eea08840b563 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:43.442674 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9586ef8-3b83-4fb4-b808-eea08840b563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:43.444268 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9586ef8-3b83-4fb4-b808-eea08840b563 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:43.461696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-f9586ef8-3b83-4fb4-b808-eea08840b563 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:43.461696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:43.461696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:43.461696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:43.461696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:43.461696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:43.461696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:43.461696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:43.461696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:43.461696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:43.461696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:43.461696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:43.461696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:43.461696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:43.461696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:43.462802 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:43.462802 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:43.462802 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:43.462802 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:43.462802 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:43.462802 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:43.462802 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:43.462802 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:43.462802 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:43.462802 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:43.462802 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:43.462802 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:43.462802 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:43.462802 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:43.462802 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:43.462802 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:43.462802 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:43.463820 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:43.463820 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:43.463820 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:43.463820 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:43.465562 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 410/818] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:43.466510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9586ef8-3b83-4fb4-b808-eea08840b563 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:43.499085 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76a8a3-0e0c-4d4c-8781-2ffb757027b8 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:43.532483 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1435904-db58-4aa7-b3e9-3caf69409164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:43.533262 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1435904-db58-4aa7-b3e9-3caf69409164 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:43.533539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1435904-db58-4aa7-b3e9-3caf69409164 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:43.533807 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1435904-db58-4aa7-b3e9-3caf69409164 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:43.540383 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1435904-db58-4aa7-b3e9-3caf69409164 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:43.541854 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1435904-db58-4aa7-b3e9-3caf69409164 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:43.556386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-c1435904-db58-4aa7-b3e9-3caf69409164 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:43.556386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:43.556386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:43.556386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:43.556386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:43.556386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:43.556386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:43.556386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:43.556386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:43.556386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:43.556386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:43.556386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:43.556386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:43.556386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:43.556386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:43.557494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:43.557494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:43.557494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:43.557494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:43.557494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:43.557494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:43.557494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:43.557494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:43.557494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:43.557494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:43.557494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:43.557494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:43.557494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:43.557494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:43.557494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:43.557494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:43.557494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:43.558650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:43.558650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:43.558650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:43.558650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:43.558650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:43.558650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:43.558650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:43.558650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:43.562307 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 409/819] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:43.563130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1435904-db58-4aa7-b3e9-3caf69409164 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:43.577052 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9586ef8-3b83-4fb4-b808-eea08840b563 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:43.610783 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ae2df-d082-446d-bec0-5f3ea74fb161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:43.611809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ae2df-d082-446d-bec0-5f3ea74fb161 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:43.612275 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ae2df-d082-446d-bec0-5f3ea74fb161 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:43.612698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ae2df-d082-446d-bec0-5f3ea74fb161 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:43.613539 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ae2df-d082-446d-bec0-5f3ea74fb161 None admin] RBAC: Adding query filter params (name=network) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:43.614652 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ae2df-d082-446d-bec0-5f3ea74fb161 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:43.616278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ae2df-d082-446d-bec0-5f3ea74fb161 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:43.621259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 411/820] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 9 22:49:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:43.622254 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ae2df-d082-446d-bec0-5f3ea74fb161 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:43.671549 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1435904-db58-4aa7-b3e9-3caf69409164 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:43.711073 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6896532-0ee9-4864-83b2-e8e16c6dffd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:43.712335 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6896532-0ee9-4864-83b2-e8e16c6dffd5 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:43.713038 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6896532-0ee9-4864-83b2-e8e16c6dffd5 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:43.713415 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6896532-0ee9-4864-83b2-e8e16c6dffd5 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:43.714399 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6896532-0ee9-4864-83b2-e8e16c6dffd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:43.715032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6896532-0ee9-4864-83b2-e8e16c6dffd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:43.716928 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6896532-0ee9-4864-83b2-e8e16c6dffd5 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:43.734522 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 410/821] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:43.735591 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6896532-0ee9-4864-83b2-e8e16c6dffd5 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:43.759413 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ae2df-d082-446d-bec0-5f3ea74fb161 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:43.793968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5d802-bac9-43a3-adca-d8843ad5d3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:43.794918 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5d802-bac9-43a3-adca-d8843ad5d3e9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:43.795295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5d802-bac9-43a3-adca-d8843ad5d3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:43.795719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5d802-bac9-43a3-adca-d8843ad5d3e9 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:43.796572 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c5d802-bac9-43a3-adca-d8843ad5d3e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:43.796948 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c5d802-bac9-43a3-adca-d8843ad5d3e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:43.798496 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c5d802-bac9-43a3-adca-d8843ad5d3e9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:43.815916 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 412/822] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:43.817439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5d802-bac9-43a3-adca-d8843ad5d3e9 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:43.841164 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6896532-0ee9-4864-83b2-e8e16c6dffd5 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:43.874540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67073ff0-e2a0-4566-8bce-2a42a3c0f003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:43.875524 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67073ff0-e2a0-4566-8bce-2a42a3c0f003 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:43.875982 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67073ff0-e2a0-4566-8bce-2a42a3c0f003 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:43.876430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67073ff0-e2a0-4566-8bce-2a42a3c0f003 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:43.877328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67073ff0-e2a0-4566-8bce-2a42a3c0f003 None admin] RBAC: Adding query filter params (type=network) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:43.877829 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67073ff0-e2a0-4566-8bce-2a42a3c0f003 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:43.880024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67073ff0-e2a0-4566-8bce-2a42a3c0f003 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:43.885015 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 411/823] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 9 22:49:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:43.886090 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67073ff0-e2a0-4566-8bce-2a42a3c0f003 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:43.922864 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5d802-bac9-43a3-adca-d8843ad5d3e9 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:43.957857 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140f71c-faec-43e7-8674-14c15beb7e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:43.958685 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140f71c-faec-43e7-8674-14c15beb7e00 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:43.958962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140f71c-faec-43e7-8674-14c15beb7e00 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:43.959234 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140f71c-faec-43e7-8674-14c15beb7e00 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:43.968622 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2140f71c-faec-43e7-8674-14c15beb7e00 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:43.970159 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2140f71c-faec-43e7-8674-14c15beb7e00 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:43.985256 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67073ff0-e2a0-4566-8bce-2a42a3c0f003 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:43.989828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-2140f71c-faec-43e7-8674-14c15beb7e00 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 22:49:43.989828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:43.989828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:43.989828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:43.989828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:43.989828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:43.989828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:43.989828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:43.989828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:43.989828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:43.989828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:43.989828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:43.989828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:43.989828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:43.989828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:43.989828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:43.990976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:43.990976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:43.990976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:43.990976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:43.990976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:43.990976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:43.990976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:43.990976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:43.990976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:43.990976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:43.990976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:43.990976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:43.990976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:43.990976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:43.990976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:43.990976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:43.990976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:43.990976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:43.991937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:43.991937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:43.991937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:43.991937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:43.991937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:43.991937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:43.991937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:43.991937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:43.991937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 22:49:43.991937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:43.995737 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 413/824] 10.222.0.23 () {56 vars in 1117 bytes} [Mon May 9 22:49:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:43.996958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140f71c-faec-43e7-8674-14c15beb7e00 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:44.028457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b719f-f1dd-4437-b2c8-9902319e1753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:44.030075 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b719f-f1dd-4437-b2c8-9902319e1753 None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:44.030615 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b719f-f1dd-4437-b2c8-9902319e1753 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:44.031087 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b719f-f1dd-4437-b2c8-9902319e1753 None admin] PATH_INFO: `/v3/projects/0500e378ea374ca692e230bbd8781318/users/8cb6c294ce1842b68a6bc73ac381ed6b/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:44.033317 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b719f-f1dd-4437-b2c8-9902319e1753 None admin] RBAC: Authorizing `identity:create_grant(project_id=0500e378ea374ca692e230bbd8781318, user_id=8cb6c294ce1842b68a6bc73ac381ed6b, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:44.034992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b719f-f1dd-4437-b2c8-9902319e1753 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:44.062251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 412/825] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:49:43 2022] PUT /identity/v3/projects/0500e378ea374ca692e230bbd8781318/users/8cb6c294ce1842b68a6bc73ac381ed6b/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:49:44.063255 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b719f-f1dd-4437-b2c8-9902319e1753 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:44.117467 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140f71c-faec-43e7-8674-14c15beb7e00 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:44.156054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d06580-ad68-4990-96c5-b9fc4cca87b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:44.157123 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d06580-ad68-4990-96c5-b9fc4cca87b9 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:44.157666 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d06580-ad68-4990-96c5-b9fc4cca87b9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:44.158170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d06580-ad68-4990-96c5-b9fc4cca87b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:44.159166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d06580-ad68-4990-96c5-b9fc4cca87b9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:44.161465 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d06580-ad68-4990-96c5-b9fc4cca87b9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:44.190293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 414/826] 10.222.0.23 () {60 vars in 1144 bytes} [Mon May 9 22:49:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:49:44.244725 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b719f-f1dd-4437-b2c8-9902319e1753 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:44.290696 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932de816-9175-466b-8f21-64083a5d2146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:44.291773 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932de816-9175-466b-8f21-64083a5d2146 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:44.292292 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932de816-9175-466b-8f21-64083a5d2146 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:44.292795 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932de816-9175-466b-8f21-64083a5d2146 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:44.293853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932de816-9175-466b-8f21-64083a5d2146 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:44.294396 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932de816-9175-466b-8f21-64083a5d2146 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:44.296248 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932de816-9175-466b-8f21-64083a5d2146 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:44.317699 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 413/827] 10.222.0.23 () {56 vars in 1149 bytes} [Mon May 9 22:49:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:44.323156 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d06580-ad68-4990-96c5-b9fc4cca87b9 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:44.443498 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d06580-ad68-4990-96c5-b9fc4cca87b9 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:44.481048 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de88fef-ef3b-4abe-8a58-c758f9cff929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:44.481849 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de88fef-ef3b-4abe-8a58-c758f9cff929 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:44.482146 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de88fef-ef3b-4abe-8a58-c758f9cff929 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:44.482432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de88fef-ef3b-4abe-8a58-c758f9cff929 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:44.495972 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de88fef-ef3b-4abe-8a58-c758f9cff929 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:44.496502 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de88fef-ef3b-4abe-8a58-c758f9cff929 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:44.514523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-4de88fef-ef3b-4abe-8a58-c758f9cff929 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:44.514523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:44.514523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:44.514523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:44.514523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:44.514523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:44.514523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:44.514523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:44.514523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:44.514523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:44.514523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:44.514523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:44.514523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:44.514523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:44.514523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:44.515457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:44.515457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:44.515457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:44.515457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:44.515457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:44.515457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:44.515457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:44.515457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:44.515457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:44.515457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:44.515457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:44.515457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:44.515457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:44.515457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:44.515457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:44.515457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:44.515457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:44.516667 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:44.516667 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:44.516667 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:44.516667 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:44.516667 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 415/828] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:44.521919 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932de816-9175-466b-8f21-64083a5d2146 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:44.631077 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932de816-9175-466b-8f21-64083a5d2146 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:44.663811 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d7022-feee-4751-b2e5-532ea1e66bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:44.664721 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d7022-feee-4751-b2e5-532ea1e66bb4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:44.665152 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d7022-feee-4751-b2e5-532ea1e66bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:44.665574 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d7022-feee-4751-b2e5-532ea1e66bb4 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:44.666463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d7022-feee-4751-b2e5-532ea1e66bb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:44.666944 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d7022-feee-4751-b2e5-532ea1e66bb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:44.668514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d7022-feee-4751-b2e5-532ea1e66bb4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:44.680957 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 414/829] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:44.685965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de88fef-ef3b-4abe-8a58-c758f9cff929 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:44.804691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de88fef-ef3b-4abe-8a58-c758f9cff929 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:44.840854 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4164dd6-e31b-44fd-b209-d58c9e6288d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:44.841651 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4164dd6-e31b-44fd-b209-d58c9e6288d1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:44.841932 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4164dd6-e31b-44fd-b209-d58c9e6288d1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:44.842251 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4164dd6-e31b-44fd-b209-d58c9e6288d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:44.849081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4164dd6-e31b-44fd-b209-d58c9e6288d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:44.850665 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4164dd6-e31b-44fd-b209-d58c9e6288d1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:44.865627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-e4164dd6-e31b-44fd-b209-d58c9e6288d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:44.865627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:44.865627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:44.865627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:44.865627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:44.865627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:44.865627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:44.865627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:44.865627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:44.865627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:44.865627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:44.865627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:44.865627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:44.865627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:44.865627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:44.866586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:44.866586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:44.866586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:44.866586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:44.866586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:44.866586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:44.866586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:44.866586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:44.866586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:44.866586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:44.866586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:44.866586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:44.866586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:44.866586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:44.866586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:44.866586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:44.866586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:44.867908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:44.867908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:44.867908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:44.867908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:44.867908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:44.867908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:44.867908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:44.867908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:44.867908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 416/830] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:44.872604 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d7022-feee-4751-b2e5-532ea1e66bb4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:44.972509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d7022-feee-4751-b2e5-532ea1e66bb4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:44.977819 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49165c88-3783-4007-88d1-cb62b4083099 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:44.978104 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49165c88-3783-4007-88d1-cb62b4083099 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:44.978405 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49165c88-3783-4007-88d1-cb62b4083099 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:44.979994 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 417/831] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:44.985336 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553caf0-b4ed-4a12-a4de-7945652076a3 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:44.985601 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553caf0-b4ed-4a12-a4de-7945652076a3 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:44.985877 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553caf0-b4ed-4a12-a4de-7945652076a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:45.002826 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-1553caf0-b4ed-4a12-a4de-7945652076a3 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:45.004007 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1553caf0-b4ed-4a12-a4de-7945652076a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:45.008919 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68424fc8-9a3a-492b-8d5c-93c0fceb1a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:45.009578 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68424fc8-9a3a-492b-8d5c-93c0fceb1a64 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:45.009856 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68424fc8-9a3a-492b-8d5c-93c0fceb1a64 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:45.010159 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68424fc8-9a3a-492b-8d5c-93c0fceb1a64 None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:45.010920 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68424fc8-9a3a-492b-8d5c-93c0fceb1a64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:45.011221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68424fc8-9a3a-492b-8d5c-93c0fceb1a64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:45.012634 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68424fc8-9a3a-492b-8d5c-93c0fceb1a64 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:45.025245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 415/832] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:45.026497 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4826a3-2f90-41e3-ba52-e26693d3333d None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:45.026777 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4826a3-2f90-41e3-ba52-e26693d3333d None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:45.027047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4826a3-2f90-41e3-ba52-e26693d3333d None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:45.030156 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 416/833] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:45.030660 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4826a3-2f90-41e3-ba52-e26693d3333d None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:45.073303 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 418/834] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 22:49:45.074965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffdc158-29f2-4a2e-b78c-d94565575419 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:45.075399 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffdc158-29f2-4a2e-b78c-d94565575419 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:45.075818 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffdc158-29f2-4a2e-b78c-d94565575419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:45.096445 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-cffdc158-29f2-4a2e-b78c-d94565575419 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:45.097641 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-cffdc158-29f2-4a2e-b78c-d94565575419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:45.132176 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4826a3-2f90-41e3-ba52-e26693d3333d None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:45.162634 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 419/835] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:45 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:49:45.163747 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffdc158-29f2-4a2e-b78c-d94565575419 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:45.178348 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88bacea-d816-45b3-9ee5-5f9d06318abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:45.179500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bacea-d816-45b3-9ee5-5f9d06318abb None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:45.179777 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bacea-d816-45b3-9ee5-5f9d06318abb None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:45.180049 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bacea-d816-45b3-9ee5-5f9d06318abb None admin] PATH_INFO: `/v3/projects/0500e378ea374ca692e230bbd8781318/users/1a6f50dcebf146baae8a9f0b5b112795/roles/44847d50e6cb4bc7938c59b4e2d99833` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:45.196137 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88bacea-d816-45b3-9ee5-5f9d06318abb None admin] RBAC: Authorizing `identity:create_grant(project_id=0500e378ea374ca692e230bbd8781318, user_id=1a6f50dcebf146baae8a9f0b5b112795, role_id=44847d50e6cb4bc7938c59b4e2d99833)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:45.197570 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88bacea-d816-45b3-9ee5-5f9d06318abb None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:45.210943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 417/836] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:49:45 2022] PUT /identity/v3/projects/0500e378ea374ca692e230bbd8781318/users/1a6f50dcebf146baae8a9f0b5b112795/roles/44847d50e6cb4bc7938c59b4e2d99833 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 22:49:45.211718 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88bacea-d816-45b3-9ee5-5f9d06318abb None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:45.295047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffdc158-29f2-4a2e-b78c-d94565575419 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:45.334559 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732cedf8-aa52-4201-a3c6-72edc28bc3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:45.335323 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cedf8-aa52-4201-a3c6-72edc28bc3ac None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:45.335573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cedf8-aa52-4201-a3c6-72edc28bc3ac None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:45.335846 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cedf8-aa52-4201-a3c6-72edc28bc3ac None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:45.336504 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732cedf8-aa52-4201-a3c6-72edc28bc3ac None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:45.337882 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732cedf8-aa52-4201-a3c6-72edc28bc3ac None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:45.348427 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88bacea-d816-45b3-9ee5-5f9d06318abb None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:45.351118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-732cedf8-aa52-4201-a3c6-72edc28bc3ac None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 22:49:45.351118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:45.351118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:49:45.351118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:49:45.351118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:49:45.351118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:49:45.351118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:49:45.351118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:45.351118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 22:49:45.351118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:45.351118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:49:45.351118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:45.351118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:45.351118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:45.351118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:45.351118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:45.352113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:45.352113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:45.352113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:45.352113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:45.352113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:45.352113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:45.352113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:45.352113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:49:45.352113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:49:45.352113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:49:45.352113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:49:45.352113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:45.352113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:45.352113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:45.352113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:45.352113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:45.352955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:45.352955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:45.352955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:45.352955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:45.352955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:45.352955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:45.352955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:45.352955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:45.352955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:45.352955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:45.352955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:45.352955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:49:45.352955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:45.352955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 22:49:45.352955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:45.352955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 420/837] 10.222.0.23 () {56 vars in 1132 bytes} [Mon May 9 22:49:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:45.359454 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732cedf8-aa52-4201-a3c6-72edc28bc3ac None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:45.391935 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d012d4-31a1-4ad1-b08d-05c8d78800d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:45.392415 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d012d4-31a1-4ad1-b08d-05c8d78800d4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:45.392817 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d012d4-31a1-4ad1-b08d-05c8d78800d4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:45.393240 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d012d4-31a1-4ad1-b08d-05c8d78800d4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:45.394002 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d012d4-31a1-4ad1-b08d-05c8d78800d4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:45.396290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d012d4-31a1-4ad1-b08d-05c8d78800d4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:45.410658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-d5d012d4-31a1-4ad1-b08d-05c8d78800d4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 22:49:45.410658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:45.410658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:49:45.410658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:49:45.410658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:49:45.410658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:49:45.410658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:49:45.410658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:45.410658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 22:49:45.410658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:45.410658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:49:45.410658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:45.410658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:45.410658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:45.410658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:45.410658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:45.410658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:45.411722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:45.411722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:45.411722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:45.411722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:45.411722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:45.411722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:45.411722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:49:45.411722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:49:45.411722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:49:45.411722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:49:45.411722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:45.411722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:45.411722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:45.411722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:45.411722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:45.411722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:45.412580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:45.412580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:45.412580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:45.412580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:45.412580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:45.412580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:45.412580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:45.412580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:45.412580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:45.412580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:45.412580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:49:45.412580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:45.412580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 22:49:45.412580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:45.412580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 418/838] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 9 22:49:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:45.421399 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d012d4-31a1-4ad1-b08d-05c8d78800d4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:45.479314 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732cedf8-aa52-4201-a3c6-72edc28bc3ac None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:45.514528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e84a70-ed1b-4edd-a2b4-c09ad45ea4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:45.515271 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e84a70-ed1b-4edd-a2b4-c09ad45ea4d5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:45.515558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e84a70-ed1b-4edd-a2b4-c09ad45ea4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:45.515838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e84a70-ed1b-4edd-a2b4-c09ad45ea4d5 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:45.516623 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e84a70-ed1b-4edd-a2b4-c09ad45ea4d5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:45.516929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e84a70-ed1b-4edd-a2b4-c09ad45ea4d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:45.518339 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e84a70-ed1b-4edd-a2b4-c09ad45ea4d5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:45.523373 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 421/839] 10.222.0.23 () {56 vars in 1126 bytes} [Mon May 9 22:49:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:45.528822 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e84a70-ed1b-4edd-a2b4-c09ad45ea4d5 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:45.535206 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d012d4-31a1-4ad1-b08d-05c8d78800d4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:45.568973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3cac4b-f7e8-4b01-96b1-6aa6ee2173cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:45.569991 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cac4b-f7e8-4b01-96b1-6aa6ee2173cc None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:45.570497 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cac4b-f7e8-4b01-96b1-6aa6ee2173cc None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:45.570971 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cac4b-f7e8-4b01-96b1-6aa6ee2173cc None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:45.571886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3cac4b-f7e8-4b01-96b1-6aa6ee2173cc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:45.572399 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3cac4b-f7e8-4b01-96b1-6aa6ee2173cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:45.573955 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3cac4b-f7e8-4b01-96b1-6aa6ee2173cc None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:45.578916 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 419/840] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 9 22:49:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:45.583896 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3cac4b-f7e8-4b01-96b1-6aa6ee2173cc None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:45.637525 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e84a70-ed1b-4edd-a2b4-c09ad45ea4d5 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:45.671184 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b36ee-f557-4938-8973-94e73aefd5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:45.672240 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b36ee-f557-4938-8973-94e73aefd5f8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:45.672706 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b36ee-f557-4938-8973-94e73aefd5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:45.673168 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b36ee-f557-4938-8973-94e73aefd5f8 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:45.674080 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b36ee-f557-4938-8973-94e73aefd5f8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:45.674664 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b36ee-f557-4938-8973-94e73aefd5f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:45.676752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b36ee-f557-4938-8973-94e73aefd5f8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:45.690455 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 422/841] 10.222.0.23 () {56 vars in 1126 bytes} [Mon May 9 22:49:45 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:45.699794 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3cac4b-f7e8-4b01-96b1-6aa6ee2173cc None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:45.737453 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e58dc-94e0-4db5-9c25-5ed952347c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:45.738143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e58dc-94e0-4db5-9c25-5ed952347c41 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:45.738565 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e58dc-94e0-4db5-9c25-5ed952347c41 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:45.739381 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e58dc-94e0-4db5-9c25-5ed952347c41 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:45.740162 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e58dc-94e0-4db5-9c25-5ed952347c41 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:45.740465 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e58dc-94e0-4db5-9c25-5ed952347c41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:45.741923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e58dc-94e0-4db5-9c25-5ed952347c41 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:45.747673 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 420/842] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 9 22:49:45 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:46.517530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26feb3e-644b-419c-b7bc-49cf700e05d8 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:46.518051 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26feb3e-644b-419c-b7bc-49cf700e05d8 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:46.518271 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26feb3e-644b-419c-b7bc-49cf700e05d8 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:46.519586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 423/843] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:46.525072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ce701-8e57-4eb2-b9d9-27331bf0a1c4 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:46.525358 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ce701-8e57-4eb2-b9d9-27331bf0a1c4 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:46.525634 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ce701-8e57-4eb2-b9d9-27331bf0a1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:46.542285 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-980ce701-8e57-4eb2-b9d9-27331bf0a1c4 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:46.544218 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-980ce701-8e57-4eb2-b9d9-27331bf0a1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:46.611740 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 421/844] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:49:46.634790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26feb3e-644b-419c-b7bc-49cf700e05d8 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:46.736429 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26feb3e-644b-419c-b7bc-49cf700e05d8 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:46.769487 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb24779-df43-4c9f-adc3-91c02de39a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:46.770311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb24779-df43-4c9f-adc3-91c02de39a9c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:46.770590 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb24779-df43-4c9f-adc3-91c02de39a9c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:46.770858 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb24779-df43-4c9f-adc3-91c02de39a9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:46.781276 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb24779-df43-4c9f-adc3-91c02de39a9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:46.789343 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-8eb24779-df43-4c9f-adc3-91c02de39a9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:49:46.789343 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:46.789343 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:46.789343 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:46.789343 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:46.789343 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:46.789343 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:46.789343 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:46.789343 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:46.789343 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:46.789343 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:46.789343 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:46.789343 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:46.789343 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:46.789343 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:46.789343 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:46.790299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:46.790299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:46.790299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:46.790299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:46.790299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:46.790299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:46.790299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:46.790299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:46.790299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:46.790299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:46.790299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:46.790299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:46.790299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:46.790299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:46.790299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:46.790299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:46.791164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:46.791164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:46.791164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:46.791164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:46.791164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:46.791164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:46.791164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:46.791164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:46.791164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:46.791164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:46.791164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:49:46.791164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:46.791164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 424/845] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:49:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:46.798184 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ce701-8e57-4eb2-b9d9-27331bf0a1c4 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:46.899519 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ce701-8e57-4eb2-b9d9-27331bf0a1c4 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:46.935989 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abefd7-4efc-4e27-bf6d-01bdfef4897c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:46.936665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abefd7-4efc-4e27-bf6d-01bdfef4897c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:46.936938 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abefd7-4efc-4e27-bf6d-01bdfef4897c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:46.937206 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abefd7-4efc-4e27-bf6d-01bdfef4897c None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:46.938008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52abefd7-4efc-4e27-bf6d-01bdfef4897c None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:46.938340 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52abefd7-4efc-4e27-bf6d-01bdfef4897c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:46.939748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52abefd7-4efc-4e27-bf6d-01bdfef4897c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:46.948460 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 422/846] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:49:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:46.955216 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb24779-df43-4c9f-adc3-91c02de39a9c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:47.057814 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb24779-df43-4c9f-adc3-91c02de39a9c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:47.092042 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a170c2-da69-4b8a-ab8a-160f3dec4d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:47.092840 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a170c2-da69-4b8a-ab8a-160f3dec4d88 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:47.093119 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a170c2-da69-4b8a-ab8a-160f3dec4d88 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:47.093395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a170c2-da69-4b8a-ab8a-160f3dec4d88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:47.104317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a170c2-da69-4b8a-ab8a-160f3dec4d88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:47.105766 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a170c2-da69-4b8a-ab8a-160f3dec4d88 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:47.123501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-b7a170c2-da69-4b8a-ab8a-160f3dec4d88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:47.123501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:47.123501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:47.123501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:47.123501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:47.123501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:47.123501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:47.123501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:47.123501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:47.123501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:47.123501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:47.123501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:47.123501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:47.123501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:47.123501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:47.124404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:47.124404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:47.124404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:47.124404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:47.124404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:47.124404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:47.124404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:47.124404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:47.124404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:47.124404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:47.124404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:47.124404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:47.124404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:47.124404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:47.124404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:47.124404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:47.124404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:47.125301 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:47.125301 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:47.125301 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:47.125301 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:47.125301 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 425/847] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:47.131325 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abefd7-4efc-4e27-bf6d-01bdfef4897c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:47.231198 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abefd7-4efc-4e27-bf6d-01bdfef4897c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:47.263733 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd697af-4bab-4213-aaf3-5b7ba2cd1ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:47.264391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd697af-4bab-4213-aaf3-5b7ba2cd1ebe None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:47.264728 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd697af-4bab-4213-aaf3-5b7ba2cd1ebe None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:47.265005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd697af-4bab-4213-aaf3-5b7ba2cd1ebe None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:47.265714 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd697af-4bab-4213-aaf3-5b7ba2cd1ebe None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:47.266014 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd697af-4bab-4213-aaf3-5b7ba2cd1ebe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:47.267401 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd697af-4bab-4213-aaf3-5b7ba2cd1ebe None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:47.280057 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 423/848] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:47.286256 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a170c2-da69-4b8a-ab8a-160f3dec4d88 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:47.394519 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a170c2-da69-4b8a-ab8a-160f3dec4d88 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:47.431252 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ff798-ef21-4583-b212-1a7787745b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:47.432409 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ff798-ef21-4583-b212-1a7787745b77 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:47.432925 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ff798-ef21-4583-b212-1a7787745b77 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:47.433400 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ff798-ef21-4583-b212-1a7787745b77 None admin] PATH_INFO: `/v3/users/swift` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:47.441284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ff798-ef21-4583-b212-1a7787745b77 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:47.442998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ff798-ef21-4583-b212-1a7787745b77 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:47.457677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-6c3ff798-ef21-4583-b212-1a7787745b77 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 22:49:47.457677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:47.457677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:47.457677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:47.457677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:47.457677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:47.457677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:47.457677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:47.457677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:47.457677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:47.457677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:47.457677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:47.457677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:47.457677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:47.457677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:47.457677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:47.458735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:47.458735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:47.458735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:47.458735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:47.458735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:47.458735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:47.458735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:47.458735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:47.458735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:47.458735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:47.458735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:47.458735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:47.458735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:47.458735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:47.458735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:47.458735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:47.458735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:47.458735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:47.459703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:47.459703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:47.459703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:47.459703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:47.459703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:47.459703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:47.459703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:47.459703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:47.459703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 22:49:47.459703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:47.466376 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 426/849] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:49:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:47.471196 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd697af-4bab-4213-aaf3-5b7ba2cd1ebe None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:47.549890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce607a83-5e58-48e7-9f6e-2e9de93d6030 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:47.550884 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce607a83-5e58-48e7-9f6e-2e9de93d6030 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:47.551953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce607a83-5e58-48e7-9f6e-2e9de93d6030 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:47.553765 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 427/850] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:49:47.559209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24807e-ccea-4d9f-820b-ea13ca9cc4fa None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:47.559771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24807e-ccea-4d9f-820b-ea13ca9cc4fa None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:47.560252 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24807e-ccea-4d9f-820b-ea13ca9cc4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:47.578414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-bf24807e-ccea-4d9f-820b-ea13ca9cc4fa None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:47.579956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-bf24807e-ccea-4d9f-820b-ea13ca9cc4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:47.582837 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd697af-4bab-4213-aaf3-5b7ba2cd1ebe None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:47.618138 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b39b3-c874-4b41-ab14-69fa4c471d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:47.618805 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b39b3-c874-4b41-ab14-69fa4c471d46 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:47.619082 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b39b3-c874-4b41-ab14-69fa4c471d46 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:47.619353 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b39b3-c874-4b41-ab14-69fa4c471d46 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:47.620183 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b39b3-c874-4b41-ab14-69fa4c471d46 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:47.620484 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b39b3-c874-4b41-ab14-69fa4c471d46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:47.624792 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b39b3-c874-4b41-ab14-69fa4c471d46 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:47.648431 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 424/851] 10.222.0.23 () {56 vars in 1135 bytes} [Mon May 9 22:49:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:47.654455 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b39b3-c874-4b41-ab14-69fa4c471d46 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:47.659188 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 428/852] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:49:47.683306 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24807e-ccea-4d9f-820b-ea13ca9cc4fa None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:47.754694 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b39b3-c874-4b41-ab14-69fa4c471d46 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:47.786691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24807e-ccea-4d9f-820b-ea13ca9cc4fa None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:47.789191 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a832f-66c7-4975-beb2-3dc2b2adc86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:47.790026 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a832f-66c7-4975-beb2-3dc2b2adc86a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:47.790474 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a832f-66c7-4975-beb2-3dc2b2adc86a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:47.790924 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a832f-66c7-4975-beb2-3dc2b2adc86a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:47.829749 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9822f-08fc-4128-83bd-db9049181e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:47.830875 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9822f-08fc-4128-83bd-db9049181e3a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:47.831311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9822f-08fc-4128-83bd-db9049181e3a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:47.831754 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9822f-08fc-4128-83bd-db9049181e3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:47.844403 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce9822f-08fc-4128-83bd-db9049181e3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:47.853317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-5ce9822f-08fc-4128-83bd-db9049181e3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:47.853317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:47.853317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:47.853317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:47.853317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:47.853317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:47.853317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:47.853317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:47.853317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:47.853317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:47.853317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:47.853317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:47.853317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:47.853317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:47.853317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:47.853317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:47.854532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:47.854532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:47.854532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:47.854532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:47.854532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:47.854532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:47.854532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:47.854532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:47.854532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:47.854532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:47.854532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:47.854532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:47.854532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:47.854532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:47.854532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:47.854532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:47.855668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:47.855668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:47.855668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:47.855668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:47.855668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:47.855668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:47.855668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:47.855668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:47.855668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:47.855668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:47.855668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 22:49:47.855668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:47.862465 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 429/853] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 9 22:49:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:47.864131 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9822f-08fc-4128-83bd-db9049181e3a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:47.866472 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7a832f-66c7-4975-beb2-3dc2b2adc86a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:47.868006 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7a832f-66c7-4975-beb2-3dc2b2adc86a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:47.969685 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9822f-08fc-4128-83bd-db9049181e3a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:47.970647 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 425/854] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:49:47 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 22:49:47.978047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a832f-66c7-4975-beb2-3dc2b2adc86a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:48.013602 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c6524-7bea-4b44-ab7d-c916a87fee62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:48.014853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c6524-7bea-4b44-ab7d-c916a87fee62 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:48.015530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c6524-7bea-4b44-ab7d-c916a87fee62 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:48.015952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c6524-7bea-4b44-ab7d-c916a87fee62 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:48.017168 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c6524-7bea-4b44-ab7d-c916a87fee62 None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:48.017783 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c6524-7bea-4b44-ab7d-c916a87fee62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:48.019364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c6524-7bea-4b44-ab7d-c916a87fee62 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:48.028908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 430/855] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 9 22:49:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:48.031176 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e1e2a-48d2-4d46-88c7-3b6136480637 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:48.031675 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e1e2a-48d2-4d46-88c7-3b6136480637 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:48.032324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e1e2a-48d2-4d46-88c7-3b6136480637 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:48.034384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 431/856] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:48.035237 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e1e2a-48d2-4d46-88c7-3b6136480637 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:48.092371 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a832f-66c7-4975-beb2-3dc2b2adc86a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:48.125378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be8a60-b4c8-467d-9b21-ec8ae28ebcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:48.126177 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be8a60-b4c8-467d-9b21-ec8ae28ebcea None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:48.126460 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be8a60-b4c8-467d-9b21-ec8ae28ebcea None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:48.126736 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be8a60-b4c8-467d-9b21-ec8ae28ebcea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:48.136790 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be8a60-b4c8-467d-9b21-ec8ae28ebcea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:48.138246 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be8a60-b4c8-467d-9b21-ec8ae28ebcea None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:48.142629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e1e2a-48d2-4d46-88c7-3b6136480637 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:48.155500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-53be8a60-b4c8-467d-9b21-ec8ae28ebcea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:48.155500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:48.155500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:48.155500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:48.155500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:48.155500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:48.155500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:48.155500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:48.155500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:48.155500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:48.155500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:48.155500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:48.155500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:48.155500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:48.155500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:48.156452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:48.156452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:48.156452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:48.156452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:48.156452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:48.156452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:48.156452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:48.156452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:48.156452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:48.156452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:48.156452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:48.156452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:48.156452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:48.156452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:48.156452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:48.156452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:48.156452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:48.157372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:48.157372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:48.157372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:48.157372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:48.157372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 426/857] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:48.157861 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034f032-59ca-43f1-a7d5-153257bcb370 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:48.158140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034f032-59ca-43f1-a7d5-153257bcb370 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:48.158399 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034f032-59ca-43f1-a7d5-153257bcb370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:48.174320 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-8034f032-59ca-43f1-a7d5-153257bcb370 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:48.175479 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8034f032-59ca-43f1-a7d5-153257bcb370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:48.180502 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b815b2-bad8-404f-98e6-c6fadbb6425f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:48.181287 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b815b2-bad8-404f-98e6-c6fadbb6425f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:48.181567 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b815b2-bad8-404f-98e6-c6fadbb6425f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:48.181838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b815b2-bad8-404f-98e6-c6fadbb6425f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:48.191941 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b815b2-bad8-404f-98e6-c6fadbb6425f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:48.193354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b815b2-bad8-404f-98e6-c6fadbb6425f None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:48.209596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-93b815b2-bad8-404f-98e6-c6fadbb6425f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:48.209596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:48.209596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:48.209596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:48.209596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:48.209596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:48.209596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:48.209596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:48.209596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:48.209596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:48.209596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:48.209596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:48.209596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:48.209596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:48.209596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:48.210530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:48.210530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:48.210530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:48.210530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:48.210530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:48.210530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:48.210530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:48.210530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:48.210530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:48.210530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:48.210530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:48.210530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:48.210530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:48.210530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:48.210530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:48.210530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:48.210530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:48.211449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:48.211449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:48.211449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:48.211449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:48.211449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 432/858] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:48.212155 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d40eea-4129-4f44-b284-c627401eb376 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:48.212254 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d40eea-4129-4f44-b284-c627401eb376 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:48.212494 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d40eea-4129-4f44-b284-c627401eb376 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:48.213743 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 433/859] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:48.214457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d40eea-4129-4f44-b284-c627401eb376 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:48.242135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 427/860] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:49:48.243057 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034f032-59ca-43f1-a7d5-153257bcb370 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:48.321568 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d40eea-4129-4f44-b284-c627401eb376 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:48.337824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034f032-59ca-43f1-a7d5-153257bcb370 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:48.352822 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c52c5-983c-424e-bff8-5a1d3bbe9b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:48.353475 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c52c5-983c-424e-bff8-5a1d3bbe9b7c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:48.353754 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c52c5-983c-424e-bff8-5a1d3bbe9b7c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:48.354027 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c52c5-983c-424e-bff8-5a1d3bbe9b7c None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:48.354767 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c52c5-983c-424e-bff8-5a1d3bbe9b7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:48.355072 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c52c5-983c-424e-bff8-5a1d3bbe9b7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:48.356434 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c52c5-983c-424e-bff8-5a1d3bbe9b7c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:48.368102 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 434/861] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:48.369372 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28a2fa-7dc3-4f88-95b3-10361f5c749e None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:48.369496 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd792ed-5816-4281-b840-0801f53878e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:48.369698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28a2fa-7dc3-4f88-95b3-10361f5c749e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:48.369940 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28a2fa-7dc3-4f88-95b3-10361f5c749e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:48.370241 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd792ed-5816-4281-b840-0801f53878e1 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:48.370600 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd792ed-5816-4281-b840-0801f53878e1 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:48.370876 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd792ed-5816-4281-b840-0801f53878e1 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:48.371594 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd792ed-5816-4281-b840-0801f53878e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:48.371897 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd792ed-5816-4281-b840-0801f53878e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:48.373249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd792ed-5816-4281-b840-0801f53878e1 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:48.385768 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 428/862] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:48.386572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd792ed-5816-4281-b840-0801f53878e1 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:48.394224 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-6a28a2fa-7dc3-4f88-95b3-10361f5c749e None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:48.395979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6a28a2fa-7dc3-4f88-95b3-10361f5c749e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:48.458605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 435/863] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:49:48.459554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28a2fa-7dc3-4f88-95b3-10361f5c749e None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:48.495884 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd792ed-5816-4281-b840-0801f53878e1 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:48.528540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb5179-738e-4e41-abd9-7ea1db25ebea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:48.529195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb5179-738e-4e41-abd9-7ea1db25ebea None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:48.529482 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb5179-738e-4e41-abd9-7ea1db25ebea None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:48.529751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb5179-738e-4e41-abd9-7ea1db25ebea None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:48.530495 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb5179-738e-4e41-abd9-7ea1db25ebea None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:48.531856 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb5179-738e-4e41-abd9-7ea1db25ebea None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:48.544442 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 429/864] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 9 22:49:48 2022] POST /identity/v3/services => generated 269 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:49:48.545232 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb5179-738e-4e41-abd9-7ea1db25ebea None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:48.576542 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28a2fa-7dc3-4f88-95b3-10361f5c749e None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:48.619272 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c26d92-a499-4e10-b3b1-66ddb6142e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:48.620076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c26d92-a499-4e10-b3b1-66ddb6142e08 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:48.620356 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c26d92-a499-4e10-b3b1-66ddb6142e08 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:48.620624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c26d92-a499-4e10-b3b1-66ddb6142e08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:48.628133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c26d92-a499-4e10-b3b1-66ddb6142e08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:48.630199 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c26d92-a499-4e10-b3b1-66ddb6142e08 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:48.646810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-f6c26d92-a499-4e10-b3b1-66ddb6142e08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:48.646810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:48.646810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:48.646810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:48.646810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:48.646810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:48.646810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:48.646810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:48.646810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:48.646810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:48.646810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:48.646810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:48.646810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:48.646810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:48.646810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:48.647821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:48.647821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:48.647821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:48.647821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:48.647821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:48.647821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:48.647821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:48.647821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:48.647821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:48.647821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:48.647821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:48.647821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:48.647821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:48.647821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:48.647821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:48.647821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:48.647821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:48.648766 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:48.648766 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:48.648766 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:48.648766 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:48.648766 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:48.648766 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:48.648766 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:48.648766 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:48.653068 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 436/865] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:48.654009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c26d92-a499-4e10-b3b1-66ddb6142e08 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:48.662606 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb5179-738e-4e41-abd9-7ea1db25ebea None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:48.704142 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d3312-f15d-41e0-8b1a-c1cfc31b2b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:48.704975 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d3312-f15d-41e0-8b1a-c1cfc31b2b14 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:48.705410 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d3312-f15d-41e0-8b1a-c1cfc31b2b14 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:48.705681 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d3312-f15d-41e0-8b1a-c1cfc31b2b14 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:48.715913 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d3312-f15d-41e0-8b1a-c1cfc31b2b14 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:48.717785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d3312-f15d-41e0-8b1a-c1cfc31b2b14 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:48.733421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-809d3312-f15d-41e0-8b1a-c1cfc31b2b14 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 22:49:48.733421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:48.733421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:48.733421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:48.733421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:48.733421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:48.733421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:48.733421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:48.733421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:48.733421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:48.733421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:48.733421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:48.733421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:48.733421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:48.733421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:48.733421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:48.734941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:48.734941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:48.734941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:48.734941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:48.734941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:48.734941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:48.734941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:48.734941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:48.734941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:48.734941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:48.734941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:48.734941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:48.734941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:48.734941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:48.734941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:48.734941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:48.734941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:48.734941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:48.735954 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:48.735954 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:48.735954 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:48.735954 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:48.735954 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:48.735954 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:48.735954 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:48.735954 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:48.735954 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 22:49:48.735954 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:48.739952 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 430/866] 10.222.0.23 () {56 vars in 1117 bytes} [Mon May 9 22:49:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:48.740842 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d3312-f15d-41e0-8b1a-c1cfc31b2b14 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:48.763900 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c26d92-a499-4e10-b3b1-66ddb6142e08 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:48.801642 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322333d-5406-41cf-af91-e04642fb64ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:48.802344 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322333d-5406-41cf-af91-e04642fb64ed None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:48.802625 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322333d-5406-41cf-af91-e04642fb64ed None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:48.802900 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322333d-5406-41cf-af91-e04642fb64ed None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:48.803675 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8322333d-5406-41cf-af91-e04642fb64ed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:48.805046 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8322333d-5406-41cf-af91-e04642fb64ed None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:48.830652 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 437/867] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 9 22:49:48 2022] POST /identity/v3/services => generated 251 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:49:48.831656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322333d-5406-41cf-af91-e04642fb64ed None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:48.866393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d3312-f15d-41e0-8b1a-c1cfc31b2b14 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:48.922457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49293c-508e-4e21-b2d1-43d7b25b745d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:48.923134 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49293c-508e-4e21-b2d1-43d7b25b745d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:48.923590 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49293c-508e-4e21-b2d1-43d7b25b745d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:48.923874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49293c-508e-4e21-b2d1-43d7b25b745d None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:48.925136 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49293c-508e-4e21-b2d1-43d7b25b745d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:48.925449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49293c-508e-4e21-b2d1-43d7b25b745d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:48.928087 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49293c-508e-4e21-b2d1-43d7b25b745d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:48.945021 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322333d-5406-41cf-af91-e04642fb64ed None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:48.945549 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 431/868] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:48.955296 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49293c-508e-4e21-b2d1-43d7b25b745d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:48.987985 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f7713d-1fd2-4cc5-8c1e-049ef5088e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:48.989387 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7713d-1fd2-4cc5-8c1e-049ef5088e26 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:48.989704 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7713d-1fd2-4cc5-8c1e-049ef5088e26 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:48.989980 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7713d-1fd2-4cc5-8c1e-049ef5088e26 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:48.990858 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f7713d-1fd2-4cc5-8c1e-049ef5088e26 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:48.991167 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f7713d-1fd2-4cc5-8c1e-049ef5088e26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:48.992693 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f7713d-1fd2-4cc5-8c1e-049ef5088e26 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:49.017199 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 438/869] 10.222.0.23 () {56 vars in 1149 bytes} [Mon May 9 22:49:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:49.018587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f7713d-1fd2-4cc5-8c1e-049ef5088e26 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:49.064918 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49293c-508e-4e21-b2d1-43d7b25b745d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:49.104211 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f14c8e4-1047-4eb8-83e3-d185f45d0aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:49.104872 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14c8e4-1047-4eb8-83e3-d185f45d0aca None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:49.105154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14c8e4-1047-4eb8-83e3-d185f45d0aca None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:49.105428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14c8e4-1047-4eb8-83e3-d185f45d0aca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:49.106282 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f14c8e4-1047-4eb8-83e3-d185f45d0aca None admin] RBAC: Adding query filter params (role.id=bda550492f4f4c5c83f52fa89fc18a7d, scope.project.id=0500e378ea374ca692e230bbd8781318, user.id=8cb6c294ce1842b68a6bc73ac381ed6b) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:49.106591 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f14c8e4-1047-4eb8-83e3-d185f45d0aca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:49.108025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f14c8e4-1047-4eb8-83e3-d185f45d0aca None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:49.115354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 432/870] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:49:48 2022] GET /identity/v3/role_assignments?user.id=8cb6c294ce1842b68a6bc73ac381ed6b&scope.project.id=0500e378ea374ca692e230bbd8781318&role.id=bda550492f4f4c5c83f52fa89fc18a7d => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:49.139603 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f7713d-1fd2-4cc5-8c1e-049ef5088e26 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:49.178382 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f99bd1-96fb-4669-bcec-ebee2b88f3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:49.179220 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f99bd1-96fb-4669-bcec-ebee2b88f3e2 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:49.179627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f99bd1-96fb-4669-bcec-ebee2b88f3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:49.180001 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f99bd1-96fb-4669-bcec-ebee2b88f3e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:49.255939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f99bd1-96fb-4669-bcec-ebee2b88f3e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:49.257612 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f99bd1-96fb-4669-bcec-ebee2b88f3e2 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:49.360979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 439/871] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:49:49 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 22:49:49.366981 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f14c8e4-1047-4eb8-83e3-d185f45d0aca None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:49.474092 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f14c8e4-1047-4eb8-83e3-d185f45d0aca None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:49.510840 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0729d4-bb5f-47a2-bb2b-4567967f9d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:49.511635 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0729d4-bb5f-47a2-bb2b-4567967f9d19 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:49.511913 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0729d4-bb5f-47a2-bb2b-4567967f9d19 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:49.512184 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0729d4-bb5f-47a2-bb2b-4567967f9d19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:49.523291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0729d4-bb5f-47a2-bb2b-4567967f9d19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:49.524726 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0729d4-bb5f-47a2-bb2b-4567967f9d19 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:49.541943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-5c0729d4-bb5f-47a2-bb2b-4567967f9d19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:49.541943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:49.541943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:49.541943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:49.541943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:49.541943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:49.541943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:49.541943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:49.541943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:49.541943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:49.541943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:49.541943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:49.541943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:49.541943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:49.541943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:49.543065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:49.543065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:49.543065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:49.543065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:49.543065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:49.543065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:49.543065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:49.543065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:49.543065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:49.543065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:49.543065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:49.543065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:49.543065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:49.543065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:49.543065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:49.543065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:49.543065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:49.544262 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:49.544262 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:49.544262 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:49.544262 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:49.544262 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 433/872] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:49.552111 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f99bd1-96fb-4669-bcec-ebee2b88f3e2 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:49.650720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f99bd1-96fb-4669-bcec-ebee2b88f3e2 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:49.686081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042d15c-cee0-426c-a1cf-d36ca5bdcd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:49.687057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042d15c-cee0-426c-a1cf-d36ca5bdcd91 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:49.687495 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042d15c-cee0-426c-a1cf-d36ca5bdcd91 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:49.687913 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042d15c-cee0-426c-a1cf-d36ca5bdcd91 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:49.688783 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b042d15c-cee0-426c-a1cf-d36ca5bdcd91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:49.689232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b042d15c-cee0-426c-a1cf-d36ca5bdcd91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:49.690837 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b042d15c-cee0-426c-a1cf-d36ca5bdcd91 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:49.707558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 440/873] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:49.709203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0729d4-bb5f-47a2-bb2b-4567967f9d19 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:49.814663 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0729d4-bb5f-47a2-bb2b-4567967f9d19 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:49.849174 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a30c10-ec1f-49d0-a6bd-93843c75d94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:49.850271 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a30c10-ec1f-49d0-a6bd-93843c75d94a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:49.850705 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a30c10-ec1f-49d0-a6bd-93843c75d94a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:49.851130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a30c10-ec1f-49d0-a6bd-93843c75d94a None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:49.858047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a30c10-ec1f-49d0-a6bd-93843c75d94a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:49.859920 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a30c10-ec1f-49d0-a6bd-93843c75d94a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:49.874391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-37a30c10-ec1f-49d0-a6bd-93843c75d94a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:49.874391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:49.874391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:49.874391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:49.874391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:49.874391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:49.874391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:49.874391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:49.874391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:49.874391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:49.874391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:49.874391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:49.874391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:49.874391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:49.874391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:49.875490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:49.875490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:49.875490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:49.875490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:49.875490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:49.875490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:49.875490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:49.875490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:49.875490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:49.875490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:49.875490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:49.875490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:49.875490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:49.875490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:49.875490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:49.875490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:49.875490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:49.876658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:49.876658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:49.876658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:49.876658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:49.876658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:49.876658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:49.876658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:49.876658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:49.883771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 434/874] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:49.886923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042d15c-cee0-426c-a1cf-d36ca5bdcd91 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:49.989350 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042d15c-cee0-426c-a1cf-d36ca5bdcd91 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:50.023907 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eeb964-c2cb-46dd-98d5-23abbb2658c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:50.024933 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eeb964-c2cb-46dd-98d5-23abbb2658c1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:50.025418 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eeb964-c2cb-46dd-98d5-23abbb2658c1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:50.025906 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eeb964-c2cb-46dd-98d5-23abbb2658c1 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:50.026927 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eeb964-c2cb-46dd-98d5-23abbb2658c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:50.027438 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eeb964-c2cb-46dd-98d5-23abbb2658c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:50.029076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eeb964-c2cb-46dd-98d5-23abbb2658c1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:50.046224 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 441/875] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:50.048236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a30c10-ec1f-49d0-a6bd-93843c75d94a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:50.152951 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a30c10-ec1f-49d0-a6bd-93843c75d94a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:50.187438 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9601e91-253b-4cf9-80d9-c5c441935e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:50.188533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9601e91-253b-4cf9-80d9-c5c441935e6b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:50.189018 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9601e91-253b-4cf9-80d9-c5c441935e6b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:50.189509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9601e91-253b-4cf9-80d9-c5c441935e6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:50.190572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9601e91-253b-4cf9-80d9-c5c441935e6b None admin] RBAC: Adding query filter params (role.id=44847d50e6cb4bc7938c59b4e2d99833, scope.project.id=0500e378ea374ca692e230bbd8781318, user.id=1a6f50dcebf146baae8a9f0b5b112795) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:50.191135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9601e91-253b-4cf9-80d9-c5c441935e6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:50.193348 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9601e91-253b-4cf9-80d9-c5c441935e6b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:50.206288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 435/876] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:49:50 2022] GET /identity/v3/role_assignments?user.id=1a6f50dcebf146baae8a9f0b5b112795&scope.project.id=0500e378ea374ca692e230bbd8781318&role.id=44847d50e6cb4bc7938c59b4e2d99833 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:50.863782 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d418593-8f23-481a-8681-31d9509407c3 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:50.864301 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d418593-8f23-481a-8681-31d9509407c3 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:50.864301 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d418593-8f23-481a-8681-31d9509407c3 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:50.865886 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 442/877] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:50.871277 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e23495-c229-4053-99a0-e6b453fda21a None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:50.871670 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e23495-c229-4053-99a0-e6b453fda21a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:50.871836 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e23495-c229-4053-99a0-e6b453fda21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:50.888754 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d5e23495-c229-4053-99a0-e6b453fda21a None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:50.890287 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d5e23495-c229-4053-99a0-e6b453fda21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:50.959475 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 436/878] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:50 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 22:49:50.981812 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d418593-8f23-481a-8681-31d9509407c3 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:51.089738 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d418593-8f23-481a-8681-31d9509407c3 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:51.124525 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db32ff-de20-4a77-974e-42089f4e8363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:51.125731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db32ff-de20-4a77-974e-42089f4e8363 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:51.126240 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db32ff-de20-4a77-974e-42089f4e8363 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:51.126720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db32ff-de20-4a77-974e-42089f4e8363 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:51.127569 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db32ff-de20-4a77-974e-42089f4e8363 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:51.129704 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db32ff-de20-4a77-974e-42089f4e8363 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:51.142446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-55db32ff-de20-4a77-974e-42089f4e8363 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 22:49:51.142446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:51.142446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:49:51.142446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:49:51.142446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:49:51.142446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:49:51.142446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:49:51.142446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:51.142446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 22:49:51.142446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:51.142446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:49:51.142446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:51.142446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:51.142446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:51.142446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:51.142446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:51.143385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:51.143385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:51.143385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:51.143385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:51.143385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:51.143385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:51.143385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:51.143385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:49:51.143385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:49:51.143385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:49:51.143385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:49:51.143385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:51.143385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:51.143385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:51.143385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:51.143385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:51.144235 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:51.144235 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:51.144235 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:51.144235 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:51.144235 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:51.144235 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:51.144235 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:51.144235 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:51.144235 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:51.144235 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:51.144235 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:51.144235 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:49:51.144235 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:51.144235 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 22:49:51.144235 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:51.147934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 443/879] 10.222.0.23 () {56 vars in 1132 bytes} [Mon May 9 22:49:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:51.153594 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e23495-c229-4053-99a0-e6b453fda21a None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:51.200358 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3f2a8-ae90-49c3-a419-a66c49ef9371 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:51.200863 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3f2a8-ae90-49c3-a419-a66c49ef9371 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:51.201376 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3f2a8-ae90-49c3-a419-a66c49ef9371 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:51.203276 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 444/880] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:51.210079 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ceb26-9eb9-44b9-bb48-55313feeb6b5 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:51.210464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ceb26-9eb9-44b9-bb48-55313feeb6b5 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:51.210819 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ceb26-9eb9-44b9-bb48-55313feeb6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:51.231734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-455ceb26-9eb9-44b9-bb48-55313feeb6b5 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:51.233378 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-455ceb26-9eb9-44b9-bb48-55313feeb6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:51.258522 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e23495-c229-4053-99a0-e6b453fda21a None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:51.297035 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138ab29-e37a-4898-abf4-728d605c5318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:51.298091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138ab29-e37a-4898-abf4-728d605c5318 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:51.298618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138ab29-e37a-4898-abf4-728d605c5318 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:51.299098 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138ab29-e37a-4898-abf4-728d605c5318 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:51.300020 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138ab29-e37a-4898-abf4-728d605c5318 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:51.300586 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138ab29-e37a-4898-abf4-728d605c5318 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:51.302261 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138ab29-e37a-4898-abf4-728d605c5318 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:51.307387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 437/881] 10.222.0.23 () {56 vars in 1126 bytes} [Mon May 9 22:49:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:51.313025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138ab29-e37a-4898-abf4-728d605c5318 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:51.322540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 445/882] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 22:49:51.345116 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ceb26-9eb9-44b9-bb48-55313feeb6b5 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:51.418099 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138ab29-e37a-4898-abf4-728d605c5318 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:51.457118 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbcc64-3a53-43b0-b450-21e5fbdacc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:51.457819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbcc64-3a53-43b0-b450-21e5fbdacc4e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:51.458101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbcc64-3a53-43b0-b450-21e5fbdacc4e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:51.458403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbcc64-3a53-43b0-b450-21e5fbdacc4e None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:51.459441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bbcc64-3a53-43b0-b450-21e5fbdacc4e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:51.459441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bbcc64-3a53-43b0-b450-21e5fbdacc4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:51.460772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bbcc64-3a53-43b0-b450-21e5fbdacc4e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:51.465836 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 438/883] 10.222.0.23 () {56 vars in 1126 bytes} [Mon May 9 22:49:51 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:51.467111 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdeb496-81a2-49b5-bc23-74fb460035c6 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:51.467397 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdeb496-81a2-49b5-bc23-74fb460035c6 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:51.467673 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdeb496-81a2-49b5-bc23-74fb460035c6 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:51.469052 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 439/884] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:51.471379 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdeb496-81a2-49b5-bc23-74fb460035c6 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:51.480473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ceb26-9eb9-44b9-bb48-55313feeb6b5 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:51.517538 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b989960-b0cb-4a32-b1c8-238f27086ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:51.518357 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b989960-b0cb-4a32-b1c8-238f27086ce8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:51.518634 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b989960-b0cb-4a32-b1c8-238f27086ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:51.518904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b989960-b0cb-4a32-b1c8-238f27086ce8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:51.519548 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b989960-b0cb-4a32-b1c8-238f27086ce8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:51.520939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b989960-b0cb-4a32-b1c8-238f27086ce8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:51.533097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-4b989960-b0cb-4a32-b1c8-238f27086ce8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 22:49:51.533097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:51.533097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:49:51.533097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:49:51.533097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:49:51.533097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:49:51.533097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:49:51.533097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:51.533097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 22:49:51.533097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:51.533097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:49:51.533097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:51.533097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:51.533097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:51.533097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:51.533097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:51.533097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:51.534324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:51.534324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:51.534324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:51.534324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:51.534324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:51.534324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:51.534324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:49:51.534324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:49:51.534324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:49:51.534324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:49:51.534324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:51.534324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:51.534324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:51.534324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:51.534324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:51.534324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:51.535189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:51.535189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:51.535189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:51.535189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:51.535189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:51.535189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:51.535189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:51.535189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:51.535189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:51.535189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:51.535189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:49:51.535189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:51.535189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 22:49:51.535189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:51.535189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 446/885] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 9 22:49:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:51.536064 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f03f2-71fb-4c6d-828d-4580da9e45e2 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:51.536064 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f03f2-71fb-4c6d-828d-4580da9e45e2 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:51.536064 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f03f2-71fb-4c6d-828d-4580da9e45e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:51.552444 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-fe5f03f2-71fb-4c6d-828d-4580da9e45e2 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:51.553619 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-fe5f03f2-71fb-4c6d-828d-4580da9e45e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:51.576321 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdeb496-81a2-49b5-bc23-74fb460035c6 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:51.610753 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd6911-3a7d-43f0-a059-5fcb82c81172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:51.611405 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd6911-3a7d-43f0-a059-5fcb82c81172 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:51.611682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd6911-3a7d-43f0-a059-5fcb82c81172 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:51.611954 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd6911-3a7d-43f0-a059-5fcb82c81172 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:51.612731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd6911-3a7d-43f0-a059-5fcb82c81172 None admin] RBAC: Adding query filter params (interface=public, service_id=01cc37be6816443dbd19fe1d5d6dde65, region_id=RegionOne) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:51.613037 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd6911-3a7d-43f0-a059-5fcb82c81172 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:51.614457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd6911-3a7d-43f0-a059-5fcb82c81172 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:51.623980 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 440/886] 10.222.0.23 () {56 vars in 1251 bytes} [Mon May 9 22:49:51 2022] GET /identity/v3/endpoints?service_id=01cc37be6816443dbd19fe1d5d6dde65&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:51.623980 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd6911-3a7d-43f0-a059-5fcb82c81172 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:51.625211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 447/887] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:49:51.626152 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f03f2-71fb-4c6d-828d-4580da9e45e2 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:51.725043 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f03f2-71fb-4c6d-828d-4580da9e45e2 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:51.728882 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd6911-3a7d-43f0-a059-5fcb82c81172 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:51.760057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5a07b-2fa8-4b55-b05b-ccabf06546e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:51.760711 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5a07b-2fa8-4b55-b05b-ccabf06546e0 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:51.760992 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5a07b-2fa8-4b55-b05b-ccabf06546e0 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:51.761265 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5a07b-2fa8-4b55-b05b-ccabf06546e0 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:51.761934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5a07b-2fa8-4b55-b05b-ccabf06546e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:51.762093 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74188690-a58f-4faf-9c19-db9d645bd310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:51.762934 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74188690-a58f-4faf-9c19-db9d645bd310 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:51.763334 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5a07b-2fa8-4b55-b05b-ccabf06546e0 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:51.763512 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74188690-a58f-4faf-9c19-db9d645bd310 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:51.763921 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74188690-a58f-4faf-9c19-db9d645bd310 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:51.764762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74188690-a58f-4faf-9c19-db9d645bd310 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:51.765065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74188690-a58f-4faf-9c19-db9d645bd310 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:51.766431 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74188690-a58f-4faf-9c19-db9d645bd310 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:51.778375 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 448/888] 10.222.0.23 () {56 vars in 1087 bytes} [Mon May 9 22:49:51 2022] GET /identity/v3/services => generated 1268 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 22:49:51.778375 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5a07b-2fa8-4b55-b05b-ccabf06546e0 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:51.781763 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 441/889] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 9 22:49:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:51.788071 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74188690-a58f-4faf-9c19-db9d645bd310 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:51.887894 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5a07b-2fa8-4b55-b05b-ccabf06546e0 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:51.910011 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74188690-a58f-4faf-9c19-db9d645bd310 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:51.927675 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393dcd2-3bed-40d6-b0cc-a4be183fe1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:51.928483 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393dcd2-3bed-40d6-b0cc-a4be183fe1ab None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:51.928761 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393dcd2-3bed-40d6-b0cc-a4be183fe1ab None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:51.929029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393dcd2-3bed-40d6-b0cc-a4be183fe1ab None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:51.929683 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a393dcd2-3bed-40d6-b0cc-a4be183fe1ab None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:51.931090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a393dcd2-3bed-40d6-b0cc-a4be183fe1ab None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:51.944078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-a393dcd2-3bed-40d6-b0cc-a4be183fe1ab None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 22:49:51.944078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:51.944078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:49:51.944078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:49:51.944078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:49:51.944078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:49:51.944078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:49:51.944078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:51.944078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 22:49:51.944078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:51.944078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:49:51.944078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:51.944078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:51.944078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:51.944078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:51.944078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:51.944078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:51.945133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:51.945133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:51.945133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:51.945133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:51.945133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:51.945133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:51.945133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:49:51.945133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:49:51.945133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:49:51.945133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:49:51.945133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:51.945133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:51.945133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:51.945133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:51.945133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:51.945133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:51.945967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:51.945967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:51.945967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:51.945967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:51.945967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:51.945967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:51.945967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:51.945967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:51.945967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:51.945967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:51.945967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:49:51.945967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:51.945967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 22:49:51.945967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:51.945967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 449/890] 10.222.0.23 () {56 vars in 1126 bytes} [Mon May 9 22:49:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:51.950963 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7f522-9403-4ca1-80f3-9d297ea746e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:51.951645 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7f522-9403-4ca1-80f3-9d297ea746e2 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:51.951920 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7f522-9403-4ca1-80f3-9d297ea746e2 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:51.952190 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7f522-9403-4ca1-80f3-9d297ea746e2 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:51.952977 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7f522-9403-4ca1-80f3-9d297ea746e2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:51.953245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7f522-9403-4ca1-80f3-9d297ea746e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:51.956892 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393dcd2-3bed-40d6-b0cc-a4be183fe1ab None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:51.957191 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7f522-9403-4ca1-80f3-9d297ea746e2 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:51.962066 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 442/891] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 9 22:49:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:51.967819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7f522-9403-4ca1-80f3-9d297ea746e2 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:52.063049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393dcd2-3bed-40d6-b0cc-a4be183fe1ab None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:52.093786 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7f522-9403-4ca1-80f3-9d297ea746e2 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:52.097727 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7dc26-5f0d-45ad-b263-65a913baa150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:52.098732 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7dc26-5f0d-45ad-b263-65a913baa150 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:52.099226 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7dc26-5f0d-45ad-b263-65a913baa150 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:52.099704 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7dc26-5f0d-45ad-b263-65a913baa150 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:52.100650 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f7dc26-5f0d-45ad-b263-65a913baa150 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:52.101152 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f7dc26-5f0d-45ad-b263-65a913baa150 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:52.102773 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f7dc26-5f0d-45ad-b263-65a913baa150 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:52.112650 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 450/892] 10.222.0.23 () {56 vars in 1122 bytes} [Mon May 9 22:49:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:52.113858 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7dc26-5f0d-45ad-b263-65a913baa150 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:52.137765 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621074c7-ed90-4c39-b337-4c66762a85e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:52.138781 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621074c7-ed90-4c39-b337-4c66762a85e4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:52.139074 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621074c7-ed90-4c39-b337-4c66762a85e4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:52.139349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621074c7-ed90-4c39-b337-4c66762a85e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:52.140140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621074c7-ed90-4c39-b337-4c66762a85e4 None admin] RBAC: Adding query filter params (interface=public, service_id=67eadec93495492a9c886e1cd65170c4, region_id=RegionOne) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:52.140451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621074c7-ed90-4c39-b337-4c66762a85e4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:52.142519 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621074c7-ed90-4c39-b337-4c66762a85e4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:52.150480 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 443/893] 10.222.0.23 () {56 vars in 1251 bytes} [Mon May 9 22:49:51 2022] GET /identity/v3/endpoints?service_id=67eadec93495492a9c886e1cd65170c4&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:52.159055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621074c7-ed90-4c39-b337-4c66762a85e4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:52.232423 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7dc26-5f0d-45ad-b263-65a913baa150 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:52.260885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621074c7-ed90-4c39-b337-4c66762a85e4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:52.274523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98cd815-c1ba-4d89-adf2-49ea08a7ddee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:52.276781 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cd815-c1ba-4d89-adf2-49ea08a7ddee None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:52.277183 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cd815-c1ba-4d89-adf2-49ea08a7ddee None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:52.277467 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cd815-c1ba-4d89-adf2-49ea08a7ddee None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:52.278207 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98cd815-c1ba-4d89-adf2-49ea08a7ddee None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:52.278519 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98cd815-c1ba-4d89-adf2-49ea08a7ddee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:52.280382 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98cd815-c1ba-4d89-adf2-49ea08a7ddee None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:52.290171 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 451/894] 10.222.0.23 () {56 vars in 1122 bytes} [Mon May 9 22:49:52 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:52.308667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1bacd-a5d0-4543-b415-686ed2c5542c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:52.309668 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1bacd-a5d0-4543-b415-686ed2c5542c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:52.310098 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1bacd-a5d0-4543-b415-686ed2c5542c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:52.310576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1bacd-a5d0-4543-b415-686ed2c5542c None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:52.311427 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1bacd-a5d0-4543-b415-686ed2c5542c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:52.313145 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1bacd-a5d0-4543-b415-686ed2c5542c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:52.321051 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 444/895] 10.222.0.23 () {56 vars in 1087 bytes} [Mon May 9 22:49:52 2022] GET /identity/v3/services => generated 1268 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 22:49:52.592426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9345057-20aa-4fda-99e7-74554c92e898 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:52.592616 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9345057-20aa-4fda-99e7-74554c92e898 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:52.592894 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9345057-20aa-4fda-99e7-74554c92e898 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:52.594554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 452/896] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:52.600460 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efb579-9b9a-405e-8b4e-0f36f179b785 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:52.600754 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efb579-9b9a-405e-8b4e-0f36f179b785 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:52.601035 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efb579-9b9a-405e-8b4e-0f36f179b785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:52.619205 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-04efb579-9b9a-405e-8b4e-0f36f179b785 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:52.620388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-04efb579-9b9a-405e-8b4e-0f36f179b785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:52.688912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 445/897] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:52 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:49:52.712266 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9345057-20aa-4fda-99e7-74554c92e898 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:52.816627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9345057-20aa-4fda-99e7-74554c92e898 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:52.850445 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff0565c-2b72-4e6f-b7c3-11fb2aee1e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:52.851173 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0565c-2b72-4e6f-b7c3-11fb2aee1e19 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:52.851451 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0565c-2b72-4e6f-b7c3-11fb2aee1e19 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:52.851725 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0565c-2b72-4e6f-b7c3-11fb2aee1e19 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:52.862332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff0565c-2b72-4e6f-b7c3-11fb2aee1e19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:52.870961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-bff0565c-2b72-4e6f-b7c3-11fb2aee1e19 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 22:49:52.870961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:52.870961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:52.870961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:52.870961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:52.870961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:52.870961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:52.870961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:52.870961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:52.870961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:52.870961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:52.870961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:52.870961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:52.870961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:52.870961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:52.870961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:52.871923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:52.871923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:52.871923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:52.871923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:52.871923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:52.871923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:52.871923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:52.871923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:52.871923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:52.871923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:52.871923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:52.871923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:52.871923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:52.871923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:52.871923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:52.871923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:52.872776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:52.872776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:52.872776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:52.872776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:52.872776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:52.872776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:52.872776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:52.872776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:52.872776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:52.872776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:52.872776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 22:49:52.872776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:52.872776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 453/898] 10.222.0.23 () {56 vars in 1120 bytes} [Mon May 9 22:49:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:52.878183 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efb579-9b9a-405e-8b4e-0f36f179b785 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:52.978681 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efb579-9b9a-405e-8b4e-0f36f179b785 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:53.013473 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75cf664-9c75-4f5d-a7cf-305e6bbea6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:53.014162 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75cf664-9c75-4f5d-a7cf-305e6bbea6b9 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:53.014451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75cf664-9c75-4f5d-a7cf-305e6bbea6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:53.014727 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75cf664-9c75-4f5d-a7cf-305e6bbea6b9 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:53.015466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75cf664-9c75-4f5d-a7cf-305e6bbea6b9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:53.015771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75cf664-9c75-4f5d-a7cf-305e6bbea6b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:53.017124 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75cf664-9c75-4f5d-a7cf-305e6bbea6b9 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:53.026134 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 446/899] 10.222.0.23 () {56 vars in 1115 bytes} [Mon May 9 22:49:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:53.031686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff0565c-2b72-4e6f-b7c3-11fb2aee1e19 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:53.136239 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff0565c-2b72-4e6f-b7c3-11fb2aee1e19 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:53.170982 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06d359-3a88-4648-8c0e-bab900c65c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:53.171772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06d359-3a88-4648-8c0e-bab900c65c55 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:53.172054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06d359-3a88-4648-8c0e-bab900c65c55 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:53.172332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06d359-3a88-4648-8c0e-bab900c65c55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:53.183457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc06d359-3a88-4648-8c0e-bab900c65c55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:53.184916 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc06d359-3a88-4648-8c0e-bab900c65c55 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:53.201943 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-dc06d359-3a88-4648-8c0e-bab900c65c55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:53.201943 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:53.201943 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:53.201943 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:53.201943 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:53.201943 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:53.201943 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:53.201943 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:53.201943 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:53.201943 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:53.201943 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:53.201943 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:53.201943 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:53.201943 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:53.201943 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:53.204192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:53.204192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:53.204192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:53.204192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:53.204192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:53.204192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:53.204192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:53.204192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:53.204192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:53.204192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:53.204192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:53.204192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:53.204192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:53.204192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:53.204192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:53.204192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:53.204192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:53.206209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:53.206209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:53.206209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:53.206209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:53.206209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 454/900] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:53.209334 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75cf664-9c75-4f5d-a7cf-305e6bbea6b9 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:53.308396 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75cf664-9c75-4f5d-a7cf-305e6bbea6b9 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:53.342072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0666833-dffb-41b6-86aa-b49365c443b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:53.342748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0666833-dffb-41b6-86aa-b49365c443b1 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:53.343029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0666833-dffb-41b6-86aa-b49365c443b1 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:53.343305 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0666833-dffb-41b6-86aa-b49365c443b1 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:53.344017 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0666833-dffb-41b6-86aa-b49365c443b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:53.344321 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0666833-dffb-41b6-86aa-b49365c443b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:53.345674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0666833-dffb-41b6-86aa-b49365c443b1 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:53.357553 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 447/901] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:53.363258 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06d359-3a88-4648-8c0e-bab900c65c55 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:53.463051 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06d359-3a88-4648-8c0e-bab900c65c55 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:53.501193 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933d78c-edfa-481d-91e8-f1992bf14295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:53.502003 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933d78c-edfa-481d-91e8-f1992bf14295 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:53.502308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933d78c-edfa-481d-91e8-f1992bf14295 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:53.502585 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933d78c-edfa-481d-91e8-f1992bf14295 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:53.510881 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b933d78c-edfa-481d-91e8-f1992bf14295 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:53.512310 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b933d78c-edfa-481d-91e8-f1992bf14295 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:53.528469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-b933d78c-edfa-481d-91e8-f1992bf14295 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 22:49:53.528469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:53.528469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:53.528469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:53.528469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:53.528469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:53.528469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:53.528469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:53.528469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:53.528469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:53.528469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:53.528469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:53.528469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:53.528469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:53.528469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:53.528469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:53.530929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:53.530929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:53.530929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:53.530929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:53.530929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:53.530929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:53.530929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:53.530929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:53.530929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:53.530929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:53.530929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:53.530929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:53.530929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:53.530929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:53.530929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:53.530929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:53.530929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:53.530929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:53.533117 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:53.533117 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:53.533117 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:53.533117 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:53.533117 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:53.533117 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:53.533117 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:53.533117 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:53.533117 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 22:49:53.533117 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:53.533117 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 455/902] 10.222.0.23 () {56 vars in 1117 bytes} [Mon May 9 22:49:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:53.538679 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0666833-dffb-41b6-86aa-b49365c443b1 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:53.643767 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0666833-dffb-41b6-86aa-b49365c443b1 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:53.679224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd59b3-2362-44a5-8601-0b1a7d172696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:53.679895 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd59b3-2362-44a5-8601-0b1a7d172696 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:53.680171 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd59b3-2362-44a5-8601-0b1a7d172696 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:53.680440 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd59b3-2362-44a5-8601-0b1a7d172696 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:53.681265 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd59b3-2362-44a5-8601-0b1a7d172696 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:53.681570 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd59b3-2362-44a5-8601-0b1a7d172696 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:53.683018 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd59b3-2362-44a5-8601-0b1a7d172696 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:53.701588 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 448/903] 10.222.0.23 () {56 vars in 1149 bytes} [Mon May 9 22:49:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:53.707414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933d78c-edfa-481d-91e8-f1992bf14295 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:53.811000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933d78c-edfa-481d-91e8-f1992bf14295 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:53.844612 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce609d1-1428-4c3d-908c-e44632dfdbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:53.845596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce609d1-1428-4c3d-908c-e44632dfdbe3 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:53.846105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce609d1-1428-4c3d-908c-e44632dfdbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:53.846612 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce609d1-1428-4c3d-908c-e44632dfdbe3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:53.914343 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce609d1-1428-4c3d-908c-e44632dfdbe3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:53.916142 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce609d1-1428-4c3d-908c-e44632dfdbe3 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:54.021824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 456/904] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:49:53 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 22:49:54.027819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd59b3-2362-44a5-8601-0b1a7d172696 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:54.129959 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd59b3-2362-44a5-8601-0b1a7d172696 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:54.139750 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a6dc1-d62f-40d8-a3ac-4e062a620b40 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:54.140368 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a6dc1-d62f-40d8-a3ac-4e062a620b40 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:54.140368 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a6dc1-d62f-40d8-a3ac-4e062a620b40 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:54.144460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 457/905] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:54.148082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de3628-74ab-4849-900e-417c1a644e48 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:54.148361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de3628-74ab-4849-900e-417c1a644e48 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:54.148641 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de3628-74ab-4849-900e-417c1a644e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:54.165908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-b9de3628-74ab-4849-900e-417c1a644e48 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:54.167128 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b9de3628-74ab-4849-900e-417c1a644e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:54.169679 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e3de1-1084-427d-a067-2e741df5a5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:54.170742 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e3de1-1084-427d-a067-2e741df5a5b4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:54.171251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e3de1-1084-427d-a067-2e741df5a5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:54.171716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e3de1-1084-427d-a067-2e741df5a5b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:54.182088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e3de1-1084-427d-a067-2e741df5a5b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:54.183825 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e3de1-1084-427d-a067-2e741df5a5b4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:54.206965 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-ba9e3de1-1084-427d-a067-2e741df5a5b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:54.206965 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:54.206965 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:54.206965 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:54.206965 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:54.206965 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:54.206965 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:54.206965 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:54.206965 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:54.206965 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:54.206965 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:54.206965 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:54.206965 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:54.206965 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:54.206965 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:54.208526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:54.208526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:54.208526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:54.208526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:54.208526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:54.208526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:54.208526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:54.208526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:54.208526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:54.208526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:54.208526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:54.208526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:54.208526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:54.208526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:54.208526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:54.208526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:54.208526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:54.209472 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:54.209472 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:54.209472 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:54.209472 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:54.211836 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 449/906] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:54.217539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e3de1-1084-427d-a067-2e741df5a5b4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:54.258145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 458/907] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 22:49:54.276142 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de3628-74ab-4849-900e-417c1a644e48 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:54.323213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e3de1-1084-427d-a067-2e741df5a5b4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:54.357098 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b17a8-c216-4765-93b9-2d3fab1ed182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:54.357805 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b17a8-c216-4765-93b9-2d3fab1ed182 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:54.358085 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b17a8-c216-4765-93b9-2d3fab1ed182 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:54.358416 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b17a8-c216-4765-93b9-2d3fab1ed182 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:54.359530 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5b17a8-c216-4765-93b9-2d3fab1ed182 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:54.359711 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5b17a8-c216-4765-93b9-2d3fab1ed182 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:54.361028 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5b17a8-c216-4765-93b9-2d3fab1ed182 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:54.376810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 450/908] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:54.377808 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b17a8-c216-4765-93b9-2d3fab1ed182 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:54.387884 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de3628-74ab-4849-900e-417c1a644e48 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:54.433526 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40ebc9-a477-4bf0-8210-3c9220ad4a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:54.434808 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40ebc9-a477-4bf0-8210-3c9220ad4a20 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:54.435097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40ebc9-a477-4bf0-8210-3c9220ad4a20 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:54.435367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40ebc9-a477-4bf0-8210-3c9220ad4a20 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:54.436018 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40ebc9-a477-4bf0-8210-3c9220ad4a20 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:54.437971 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40ebc9-a477-4bf0-8210-3c9220ad4a20 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:54.453956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-be40ebc9-a477-4bf0-8210-3c9220ad4a20 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 22:49:54.453956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:54.453956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:49:54.453956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:49:54.453956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:49:54.453956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:49:54.453956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:49:54.453956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:54.453956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 22:49:54.453956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:54.453956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:49:54.453956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:54.453956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:54.453956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:54.453956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:54.453956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:54.454979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:54.454979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:54.454979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:54.454979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:54.454979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:54.454979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:54.454979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:54.454979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:49:54.454979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:49:54.454979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:49:54.454979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:49:54.454979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:54.454979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:54.454979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:54.454979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:54.454979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:54.455832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:54.455832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:54.455832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:54.455832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:54.455832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:54.455832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:54.455832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:54.455832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:54.455832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:54.455832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:54.455832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:54.455832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:49:54.455832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:54.455832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 22:49:54.455832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:54.458196 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 459/909] 10.222.0.23 () {56 vars in 1132 bytes} [Mon May 9 22:49:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:54.461569 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40ebc9-a477-4bf0-8210-3c9220ad4a20 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:54.513458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b17a8-c216-4765-93b9-2d3fab1ed182 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:54.557748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd385a8-4fc8-40be-be80-69e699ca7b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:54.559281 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd385a8-4fc8-40be-be80-69e699ca7b62 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:54.560067 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd385a8-4fc8-40be-be80-69e699ca7b62 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:54.560574 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd385a8-4fc8-40be-be80-69e699ca7b62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:54.570300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd385a8-4fc8-40be-be80-69e699ca7b62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:54.572774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd385a8-4fc8-40be-be80-69e699ca7b62 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:54.583264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40ebc9-a477-4bf0-8210-3c9220ad4a20 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:54.598047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-2fd385a8-4fc8-40be-be80-69e699ca7b62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:54.598047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:54.598047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:54.598047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:54.598047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:54.598047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:54.598047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:54.598047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:54.598047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:54.598047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:54.598047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:54.598047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:54.598047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:54.598047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:54.598047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:54.599043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:54.599043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:54.599043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:54.599043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:54.599043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:54.599043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:54.599043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:54.599043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:54.599043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:54.599043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:54.599043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:54.599043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:54.599043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:54.599043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:54.599043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:54.599043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:54.599043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:54.599939 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:54.599939 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:54.599939 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:54.599939 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:54.599939 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:54.599939 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:54.599939 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:54.599939 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:54.606436 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 451/910] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:54.606436 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4975b-9ebe-440f-b1e6-d354bb6f1adb None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:54.606436 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4975b-9ebe-440f-b1e6-d354bb6f1adb None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:54.606436 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4975b-9ebe-440f-b1e6-d354bb6f1adb None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:54.606436 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 452/911] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:54.609520 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f4975b-9ebe-440f-b1e6-d354bb6f1adb None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:54.632431 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d696e9-af35-46f6-8cf7-d8eb96f67666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:54.633189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d696e9-af35-46f6-8cf7-d8eb96f67666 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:54.636554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d696e9-af35-46f6-8cf7-d8eb96f67666 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:54.636554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d696e9-af35-46f6-8cf7-d8eb96f67666 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:54.636554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d696e9-af35-46f6-8cf7-d8eb96f67666 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:54.636554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d696e9-af35-46f6-8cf7-d8eb96f67666 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:54.636554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d696e9-af35-46f6-8cf7-d8eb96f67666 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:54.641384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 460/912] 10.222.0.23 () {56 vars in 1126 bytes} [Mon May 9 22:49:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:54.642732 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0824b23-df25-4434-abb7-2e177ed4a36c None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:54.642844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0824b23-df25-4434-abb7-2e177ed4a36c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:54.643119 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0824b23-df25-4434-abb7-2e177ed4a36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:54.660735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e0824b23-df25-4434-abb7-2e177ed4a36c None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:54.661891 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e0824b23-df25-4434-abb7-2e177ed4a36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:54.717823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f4975b-9ebe-440f-b1e6-d354bb6f1adb None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:54.726309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 461/913] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:49:54.727150 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0824b23-df25-4434-abb7-2e177ed4a36c None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:54.750931 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0082a-73f9-40b5-baa1-806058a62fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:54.751581 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0082a-73f9-40b5-baa1-806058a62fef None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:54.751858 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0082a-73f9-40b5-baa1-806058a62fef None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:54.752130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0082a-73f9-40b5-baa1-806058a62fef None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:54.752901 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b0082a-73f9-40b5-baa1-806058a62fef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:54.753203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b0082a-73f9-40b5-baa1-806058a62fef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:54.754673 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b0082a-73f9-40b5-baa1-806058a62fef None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:54.765651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 453/914] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:54.766893 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d86619-a904-407e-a746-36535c28f1d3 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:54.767177 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d86619-a904-407e-a746-36535c28f1d3 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:54.767459 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d86619-a904-407e-a746-36535c28f1d3 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:54.770160 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 454/915] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:54.770608 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d86619-a904-407e-a746-36535c28f1d3 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:54.827002 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0824b23-df25-4434-abb7-2e177ed4a36c None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:54.859994 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f64c7-4263-4a08-a152-1aed8ed0da89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:54.860642 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f64c7-4263-4a08-a152-1aed8ed0da89 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:54.860920 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f64c7-4263-4a08-a152-1aed8ed0da89 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:54.861195 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f64c7-4263-4a08-a152-1aed8ed0da89 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:54.861881 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f64c7-4263-4a08-a152-1aed8ed0da89 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:54.862208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f64c7-4263-4a08-a152-1aed8ed0da89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:54.863583 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f64c7-4263-4a08-a152-1aed8ed0da89 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:54.868103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 462/916] 10.222.0.23 () {56 vars in 1126 bytes} [Mon May 9 22:49:54 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:54.868864 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f64c7-4263-4a08-a152-1aed8ed0da89 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:54.873177 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d86619-a904-407e-a746-36535c28f1d3 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:54.905404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360b7f9-b0e4-4c50-b537-26afe08b1b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:54.905985 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360b7f9-b0e4-4c50-b537-26afe08b1b08 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:54.906291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360b7f9-b0e4-4c50-b537-26afe08b1b08 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:54.906566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360b7f9-b0e4-4c50-b537-26afe08b1b08 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:54.907270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360b7f9-b0e4-4c50-b537-26afe08b1b08 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:54.908625 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360b7f9-b0e4-4c50-b537-26afe08b1b08 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:54.920811 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 455/917] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 9 22:49:54 2022] POST /identity/v3/services => generated 241 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:49:54.922143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251408c6-7d04-4bb6-b6d8-6e4b3a9e32c1 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:54.922441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251408c6-7d04-4bb6-b6d8-6e4b3a9e32c1 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:54.922707 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251408c6-7d04-4bb6-b6d8-6e4b3a9e32c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:54.939421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-251408c6-7d04-4bb6-b6d8-6e4b3a9e32c1 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:54.940629 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-251408c6-7d04-4bb6-b6d8-6e4b3a9e32c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:54.973652 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f64c7-4263-4a08-a152-1aed8ed0da89 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:55.015753 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c055dd-d09d-4565-b932-f095f01207fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:55.016392 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c055dd-d09d-4565-b932-f095f01207fd None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:55.016670 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c055dd-d09d-4565-b932-f095f01207fd None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:55.016953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c055dd-d09d-4565-b932-f095f01207fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:55.017753 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c055dd-d09d-4565-b932-f095f01207fd None admin] RBAC: Adding query filter params (role.id=8b7c99010273443e88d3e81ce6924349, scope.project.id=0500e378ea374ca692e230bbd8781318, user.id=1a6f50dcebf146baae8a9f0b5b112795) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:55.018059 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c055dd-d09d-4565-b932-f095f01207fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:55.018628 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 456/918] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:49:55.019574 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c055dd-d09d-4565-b932-f095f01207fd None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:55.020568 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251408c6-7d04-4bb6-b6d8-6e4b3a9e32c1 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:55.030487 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 463/919] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:49:54 2022] GET /identity/v3/role_assignments?user.id=1a6f50dcebf146baae8a9f0b5b112795&scope.project.id=0500e378ea374ca692e230bbd8781318&role.id=8b7c99010273443e88d3e81ce6924349 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:55.043219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c055dd-d09d-4565-b932-f095f01207fd None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:55.153739 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251408c6-7d04-4bb6-b6d8-6e4b3a9e32c1 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:55.170808 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c055dd-d09d-4565-b932-f095f01207fd None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:55.198373 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c585e6-2df1-4cbf-a703-69604bb265c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:55.199100 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c585e6-2df1-4cbf-a703-69604bb265c0 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:55.199378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c585e6-2df1-4cbf-a703-69604bb265c0 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:55.199648 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c585e6-2df1-4cbf-a703-69604bb265c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:55.200456 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c585e6-2df1-4cbf-a703-69604bb265c0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:55.201852 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c585e6-2df1-4cbf-a703-69604bb265c0 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:55.208531 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c3cd5-9cf5-4eae-9307-52b0326c5bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:55.209422 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c3cd5-9cf5-4eae-9307-52b0326c5bc6 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:55.209717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c3cd5-9cf5-4eae-9307-52b0326c5bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:55.210000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c3cd5-9cf5-4eae-9307-52b0326c5bc6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:55.210699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c3cd5-9cf5-4eae-9307-52b0326c5bc6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:55.212098 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c3cd5-9cf5-4eae-9307-52b0326c5bc6 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:55.227993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-5b9c3cd5-9cf5-4eae-9307-52b0326c5bc6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 22:49:55.227993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:55.227993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:49:55.227993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:49:55.227993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:49:55.227993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:49:55.227993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:49:55.227993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:55.227993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 22:49:55.227993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:55.227993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:49:55.227993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:55.227993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:55.227993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:55.227993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:55.227993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:55.227993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:55.229219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:55.229219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:55.229219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:55.229219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:55.229219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:55.229219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:55.229219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:49:55.229219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:49:55.229219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:49:55.229219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:49:55.229219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:55.229219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:55.229219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:55.229219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:55.229219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:55.229219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:55.230170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:55.230170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:55.230170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:55.230170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:55.230170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:55.230170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:55.230170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:55.230170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:55.230170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:55.230170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:55.230170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:49:55.230170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:55.230170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 22:49:55.230170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:55.230170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 464/920] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 9 22:49:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:55.236336 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 457/921] 10.222.0.23 () {60 vars in 1144 bytes} [Mon May 9 22:49:55 2022] POST /identity/v3/endpoints => generated 347 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:49:55.243483 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c3cd5-9cf5-4eae-9307-52b0326c5bc6 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:55.362599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c3cd5-9cf5-4eae-9307-52b0326c5bc6 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:55.405469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f640248-c7e3-4d83-9a92-edd6b2c78c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:55.406164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f640248-c7e3-4d83-9a92-edd6b2c78c95 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:55.406459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f640248-c7e3-4d83-9a92-edd6b2c78c95 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:55.406726 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f640248-c7e3-4d83-9a92-edd6b2c78c95 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:55.407433 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f640248-c7e3-4d83-9a92-edd6b2c78c95 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:55.407739 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f640248-c7e3-4d83-9a92-edd6b2c78c95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:55.409257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f640248-c7e3-4d83-9a92-edd6b2c78c95 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:55.415792 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 465/922] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 9 22:49:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:55.421614 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c585e6-2df1-4cbf-a703-69604bb265c0 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:55.523531 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c585e6-2df1-4cbf-a703-69604bb265c0 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:55.556685 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db63245-6b90-48d7-8c47-86bf4704cc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:55.557350 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db63245-6b90-48d7-8c47-86bf4704cc12 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:55.557632 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db63245-6b90-48d7-8c47-86bf4704cc12 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:55.557909 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db63245-6b90-48d7-8c47-86bf4704cc12 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:55.558640 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db63245-6b90-48d7-8c47-86bf4704cc12 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:55.558948 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db63245-6b90-48d7-8c47-86bf4704cc12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:55.560313 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db63245-6b90-48d7-8c47-86bf4704cc12 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:55.565482 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 458/923] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 9 22:49:55 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:55.571680 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f640248-c7e3-4d83-9a92-edd6b2c78c95 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:55.672475 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f640248-c7e3-4d83-9a92-edd6b2c78c95 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:55.705562 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a203521-bb80-4efe-b2e5-37b876e11422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:55.706252 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a203521-bb80-4efe-b2e5-37b876e11422 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:55.706532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a203521-bb80-4efe-b2e5-37b876e11422 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:55.706799 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a203521-bb80-4efe-b2e5-37b876e11422 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:55.707563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a203521-bb80-4efe-b2e5-37b876e11422 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:55.708911 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a203521-bb80-4efe-b2e5-37b876e11422 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:55.730440 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 466/924] 10.222.0.23 () {60 vars in 1144 bytes} [Mon May 9 22:49:55 2022] POST /identity/v3/endpoints => generated 346 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:49:57.277190 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc162b-5d0a-44c5-8704-69c614cd9890 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:57.277747 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc162b-5d0a-44c5-8704-69c614cd9890 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:57.277747 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc162b-5d0a-44c5-8704-69c614cd9890 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:57.279435 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 459/925] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:57.284784 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3d038-659b-4d45-b1e1-53fa976b8416 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:57.285229 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3d038-659b-4d45-b1e1-53fa976b8416 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:57.285506 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3d038-659b-4d45-b1e1-53fa976b8416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:57.302197 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-8be3d038-659b-4d45-b1e1-53fa976b8416 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:57.304789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f3c9ce-0e9a-4b66-b22d-77c5351207a9 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:57.305072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f3c9ce-0e9a-4b66-b22d-77c5351207a9 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:57.305347 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f3c9ce-0e9a-4b66-b22d-77c5351207a9 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:57.306336 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-8be3d038-659b-4d45-b1e1-53fa976b8416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:57.306958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 460/926] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:49:57.319244 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee7d94-3c05-4462-b15b-c72aa215c848 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:57.319668 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee7d94-3c05-4462-b15b-c72aa215c848 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:57.320040 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee7d94-3c05-4462-b15b-c72aa215c848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:57.338310 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-fcee7d94-3c05-4462-b15b-c72aa215c848 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:57.339708 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-fcee7d94-3c05-4462-b15b-c72aa215c848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:57.387655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 467/927] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:49:57.415642 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3d038-659b-4d45-b1e1-53fa976b8416 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:57.425258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 461/928] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 22:49:57.449706 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee7d94-3c05-4462-b15b-c72aa215c848 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:57.514516 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3d038-659b-4d45-b1e1-53fa976b8416 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:57.548072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee7d94-3c05-4462-b15b-c72aa215c848 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:57.549811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac91f2-5c79-4462-a9c6-2fb77bb2e2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:57.550649 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac91f2-5c79-4462-a9c6-2fb77bb2e2dd None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:57.551453 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac91f2-5c79-4462-a9c6-2fb77bb2e2dd None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:57.551726 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac91f2-5c79-4462-a9c6-2fb77bb2e2dd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:57.552373 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac91f2-5c79-4462-a9c6-2fb77bb2e2dd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:57.553732 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac91f2-5c79-4462-a9c6-2fb77bb2e2dd None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:57.565394 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-ddac91f2-5c79-4462-a9c6-2fb77bb2e2dd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 22:49:57.565394 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:57.565394 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:49:57.565394 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:49:57.565394 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:49:57.565394 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:49:57.565394 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:49:57.565394 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:57.565394 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 22:49:57.565394 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:57.565394 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:49:57.565394 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:57.565394 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:57.565394 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:57.565394 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:57.565394 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:57.565394 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:57.567247 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:57.567247 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:57.567247 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:57.567247 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:57.567247 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:57.567247 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:57.567247 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:49:57.567247 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:49:57.567247 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:49:57.567247 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:49:57.567247 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:57.567247 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:57.567247 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:57.567247 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:57.567247 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:57.567247 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:57.568593 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:57.568593 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:57.568593 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:57.568593 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:57.568593 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:57.568593 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:57.568593 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:57.568593 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:57.568593 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:57.568593 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:57.568593 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:49:57.568593 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:57.568593 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 22:49:57.568593 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:57.568593 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 468/929] 10.222.0.23 () {56 vars in 1126 bytes} [Mon May 9 22:49:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:57.569405 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2269e-ebae-49d0-89fd-6d15dff4dae6 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:57.569405 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2269e-ebae-49d0-89fd-6d15dff4dae6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:57.569405 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2269e-ebae-49d0-89fd-6d15dff4dae6 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:57.574561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 469/930] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:57.574561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2269e-ebae-49d0-89fd-6d15dff4dae6 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:57.589402 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f7fb3-80db-4962-8b08-4d45f56934c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:57.590237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f7fb3-80db-4962-8b08-4d45f56934c4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:57.590516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f7fb3-80db-4962-8b08-4d45f56934c4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:57.590786 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f7fb3-80db-4962-8b08-4d45f56934c4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:57.600518 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2f7fb3-80db-4962-8b08-4d45f56934c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:49:57.608553 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-cd2f7fb3-80db-4962-8b08-4d45f56934c4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 22:49:57.608553 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:57.608553 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:57.608553 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:57.608553 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:57.608553 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:57.608553 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:57.608553 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:57.608553 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:57.608553 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:57.608553 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:57.608553 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:57.608553 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:49:57.608553 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:49:57.608553 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:49:57.608553 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:49:57.609789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:49:57.609789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:49:57.609789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:49:57.609789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:49:57.609789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:57.609789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:57.609789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:57.609789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:57.609789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:57.609789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:57.609789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:57.609789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:57.609789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:57.609789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:57.609789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:57.609789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:57.611264 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:57.611264 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:57.611264 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:57.611264 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:57.611264 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:49:57.611264 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:49:57.611264 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:49:57.611264 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:49:57.611264 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:49:57.611264 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:49:57.611264 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 22:49:57.611264 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:57.611264 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 462/931] 10.222.0.23 () {56 vars in 1120 bytes} [Mon May 9 22:49:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:57.611976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ed99a-ee93-4f28-8ba0-91f92dd45e4f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:57.611976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ed99a-ee93-4f28-8ba0-91f92dd45e4f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:57.612115 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ed99a-ee93-4f28-8ba0-91f92dd45e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:57.630811 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-2a1ed99a-ee93-4f28-8ba0-91f92dd45e4f None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:57.631990 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2a1ed99a-ee93-4f28-8ba0-91f92dd45e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:57.684408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2269e-ebae-49d0-89fd-6d15dff4dae6 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:57.698435 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 463/932] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:49:57.699567 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ed99a-ee93-4f28-8ba0-91f92dd45e4f None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:57.729324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6efd0-90ea-4175-bc82-8f9f22741b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:57.730076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6efd0-90ea-4175-bc82-8f9f22741b92 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:57.730385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6efd0-90ea-4175-bc82-8f9f22741b92 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:57.730656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6efd0-90ea-4175-bc82-8f9f22741b92 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:57.731352 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd6efd0-90ea-4175-bc82-8f9f22741b92 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:57.731736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd6efd0-90ea-4175-bc82-8f9f22741b92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:57.733084 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd6efd0-90ea-4175-bc82-8f9f22741b92 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:57.739412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 470/933] 10.222.0.23 () {56 vars in 1122 bytes} [Mon May 9 22:49:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:57.744193 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6efd0-90ea-4175-bc82-8f9f22741b92 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:57.797848 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ed99a-ee93-4f28-8ba0-91f92dd45e4f None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:57.831673 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e6794-d3d6-4360-bd29-07eca97421ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:57.832311 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e6794-d3d6-4360-bd29-07eca97421ed None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:57.832593 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e6794-d3d6-4360-bd29-07eca97421ed None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:57.832869 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e6794-d3d6-4360-bd29-07eca97421ed None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:57.833632 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e6794-d3d6-4360-bd29-07eca97421ed None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:57.833939 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e6794-d3d6-4360-bd29-07eca97421ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:57.835403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e6794-d3d6-4360-bd29-07eca97421ed None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:57.844307 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 464/934] 10.222.0.23 () {56 vars in 1115 bytes} [Mon May 9 22:49:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:57.845185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e6794-d3d6-4360-bd29-07eca97421ed None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:57.846659 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6efd0-90ea-4175-bc82-8f9f22741b92 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:57.885540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1cb40-0de7-41d1-9243-bd4ba68d9457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:57.886418 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1cb40-0de7-41d1-9243-bd4ba68d9457 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:57.886713 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1cb40-0de7-41d1-9243-bd4ba68d9457 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:57.886990 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1cb40-0de7-41d1-9243-bd4ba68d9457 None admin] PATH_INFO: `/v3/services/compute` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:57.887635 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1cb40-0de7-41d1-9243-bd4ba68d9457 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:57.889048 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1cb40-0de7-41d1-9243-bd4ba68d9457 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:57.901047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-11a1cb40-0de7-41d1-9243-bd4ba68d9457 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 09 22:49:57.901047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:57.901047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:49:57.901047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:49:57.901047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:49:57.901047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:49:57.901047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:49:57.901047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:57.901047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 22:49:57.901047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:57.901047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:49:57.901047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:57.901047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:57.901047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:57.901047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:57.901047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:57.901047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:57.902049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:57.902049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:57.902049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:57.902049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:57.902049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:57.902049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:57.902049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:49:57.902049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:49:57.902049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:49:57.902049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:49:57.902049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:57.902049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:57.902049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:57.902049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:57.902049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:57.902049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:57.903599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:57.903599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:57.903599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:57.903599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:57.903599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:57.903599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:57.903599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:57.903599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:57.903599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:57.903599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:57.903599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:49:57.903599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:49:57.903599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 22:49:57.903599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:57.903599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 471/935] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:57.904401 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1cb40-0de7-41d1-9243-bd4ba68d9457 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:57.952952 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e6794-d3d6-4360-bd29-07eca97421ed None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:57.986483 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf617cbb-8133-4c23-88a5-16d2e6f29d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:57.987142 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf617cbb-8133-4c23-88a5-16d2e6f29d10 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:57.987422 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf617cbb-8133-4c23-88a5-16d2e6f29d10 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:57.987697 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf617cbb-8133-4c23-88a5-16d2e6f29d10 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:57.988445 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf617cbb-8133-4c23-88a5-16d2e6f29d10 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:57.988747 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf617cbb-8133-4c23-88a5-16d2e6f29d10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:57.990148 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf617cbb-8133-4c23-88a5-16d2e6f29d10 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:57.994992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 465/936] 10.222.0.23 () {56 vars in 1122 bytes} [Mon May 9 22:49:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:57.996003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf617cbb-8133-4c23-88a5-16d2e6f29d10 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:58.010379 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1cb40-0de7-41d1-9243-bd4ba68d9457 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:58.042541 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f5366-0408-4595-b8dc-7e937e69ebed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:58.043318 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f5366-0408-4595-b8dc-7e937e69ebed None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:58.043593 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f5366-0408-4595-b8dc-7e937e69ebed None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:58.043864 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f5366-0408-4595-b8dc-7e937e69ebed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:58.053972 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f5366-0408-4595-b8dc-7e937e69ebed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:58.055422 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f5366-0408-4595-b8dc-7e937e69ebed None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:58.071939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-957f5366-0408-4595-b8dc-7e937e69ebed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:58.071939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:58.071939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:58.071939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:58.071939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:58.071939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:58.071939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:58.071939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:58.071939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:58.071939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:58.071939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:58.071939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:58.071939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:58.071939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:58.071939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:58.073144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:58.073144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:58.073144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:58.073144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:58.073144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:58.073144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:58.073144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:58.073144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:58.073144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:58.073144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:58.073144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:58.073144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:58.073144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:58.073144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:58.073144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:58.073144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:58.073144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:58.074253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:58.074253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:58.074253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:58.074253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:58.074253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 472/937] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:58.075079 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f5366-0408-4595-b8dc-7e937e69ebed None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:58.093492 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf617cbb-8133-4c23-88a5-16d2e6f29d10 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:58.124611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e69e07-04fa-4812-926c-53ee0bd822cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:58.125268 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e69e07-04fa-4812-926c-53ee0bd822cd None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:58.125548 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e69e07-04fa-4812-926c-53ee0bd822cd None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:58.125823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e69e07-04fa-4812-926c-53ee0bd822cd None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:58.126543 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e69e07-04fa-4812-926c-53ee0bd822cd None admin] RBAC: Adding query filter params (name=compute) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:58.126850 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e69e07-04fa-4812-926c-53ee0bd822cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:58.128208 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e69e07-04fa-4812-926c-53ee0bd822cd None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:58.133194 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 466/938] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 9 22:49:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:58.134397 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99e011-3041-4cc0-ba1d-82f5aa734070 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:58.134682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99e011-3041-4cc0-ba1d-82f5aa734070 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:58.134954 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99e011-3041-4cc0-ba1d-82f5aa734070 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:58.138073 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 467/939] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:49:58.138073 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99e011-3041-4cc0-ba1d-82f5aa734070 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:58.172512 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f5366-0408-4595-b8dc-7e937e69ebed None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:58.204769 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb841143-23cf-495a-b0d4-b3424d64e5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:58.205416 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb841143-23cf-495a-b0d4-b3424d64e5c8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:58.205697 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb841143-23cf-495a-b0d4-b3424d64e5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:58.205972 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb841143-23cf-495a-b0d4-b3424d64e5c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:58.206787 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb841143-23cf-495a-b0d4-b3424d64e5c8 None admin] RBAC: Adding query filter params (interface=public, service_id=31572c3ef7d948e58386d2a29f51f807, region_id=RegionOne) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:58.207126 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb841143-23cf-495a-b0d4-b3424d64e5c8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:58.208486 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb841143-23cf-495a-b0d4-b3424d64e5c8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:58.213342 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 473/940] 10.222.0.23 () {56 vars in 1251 bytes} [Mon May 9 22:49:58 2022] GET /identity/v3/endpoints?service_id=31572c3ef7d948e58386d2a29f51f807&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:58.214109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb841143-23cf-495a-b0d4-b3424d64e5c8 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:58.232796 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99e011-3041-4cc0-ba1d-82f5aa734070 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:58.265001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace9edc-76b7-4f1f-a741-684d29299782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:58.265647 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace9edc-76b7-4f1f-a741-684d29299782 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:58.265926 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace9edc-76b7-4f1f-a741-684d29299782 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:58.266229 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace9edc-76b7-4f1f-a741-684d29299782 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:58.266948 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace9edc-76b7-4f1f-a741-684d29299782 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:58.267248 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace9edc-76b7-4f1f-a741-684d29299782 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:58.268596 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace9edc-76b7-4f1f-a741-684d29299782 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:58.279894 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 468/941] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:58.280931 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00819f78-971a-4a7a-a9da-c13e537c3ec5 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:58.281212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00819f78-971a-4a7a-a9da-c13e537c3ec5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:58.281485 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00819f78-971a-4a7a-a9da-c13e537c3ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:58.298670 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-00819f78-971a-4a7a-a9da-c13e537c3ec5 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:49:58.299842 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-00819f78-971a-4a7a-a9da-c13e537c3ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:49:58.310698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb841143-23cf-495a-b0d4-b3424d64e5c8 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:58.341957 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5e67d-3369-45ef-bee7-6d6b4f46721b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:58.342629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5e67d-3369-45ef-bee7-6d6b4f46721b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:58.342904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5e67d-3369-45ef-bee7-6d6b4f46721b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:58.343171 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5e67d-3369-45ef-bee7-6d6b4f46721b None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:58.343905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5e67d-3369-45ef-bee7-6d6b4f46721b None admin] RBAC: Adding query filter params (type=compute) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:58.344156 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5e67d-3369-45ef-bee7-6d6b4f46721b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:58.345504 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5e67d-3369-45ef-bee7-6d6b4f46721b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:58.349868 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 474/942] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 9 22:49:58 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:58.350583 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5e67d-3369-45ef-bee7-6d6b4f46721b None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:58.365596 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 469/943] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:49:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:49:58.366505 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00819f78-971a-4a7a-a9da-c13e537c3ec5 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:58.462885 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5e67d-3369-45ef-bee7-6d6b4f46721b None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:58.471969 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00819f78-971a-4a7a-a9da-c13e537c3ec5 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:58.499481 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b22a04-8ce2-4089-a6ea-e4060d11787a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:58.500156 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b22a04-8ce2-4089-a6ea-e4060d11787a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:58.500435 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b22a04-8ce2-4089-a6ea-e4060d11787a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:58.500708 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b22a04-8ce2-4089-a6ea-e4060d11787a None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:58.501393 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b22a04-8ce2-4089-a6ea-e4060d11787a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:58.502809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b22a04-8ce2-4089-a6ea-e4060d11787a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:58.509777 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 475/944] 10.222.0.23 () {56 vars in 1087 bytes} [Mon May 9 22:49:58 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 22:49:58.510583 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b22a04-8ce2-4089-a6ea-e4060d11787a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:58.511968 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de0942-edf4-42c5-9f2f-2e38c732c882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:58.513046 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de0942-edf4-42c5-9f2f-2e38c732c882 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:58.513793 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de0942-edf4-42c5-9f2f-2e38c732c882 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:58.514390 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de0942-edf4-42c5-9f2f-2e38c732c882 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:58.525844 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de0942-edf4-42c5-9f2f-2e38c732c882 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:58.527967 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de0942-edf4-42c5-9f2f-2e38c732c882 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:58.546494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-f5de0942-edf4-42c5-9f2f-2e38c732c882 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 22:49:58.546494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:58.546494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:58.546494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:58.546494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:58.546494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:58.546494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:58.546494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:58.546494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:58.546494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:58.546494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:58.546494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:58.546494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:49:58.546494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:49:58.546494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:49:58.546494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:49:58.547540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:58.547540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:58.547540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:49:58.547540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:58.547540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:49:58.547540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:49:58.547540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:58.547540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:58.547540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:58.547540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:58.547540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:58.547540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:58.547540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:58.547540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:58.547540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:58.547540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:58.547540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:58.547540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:58.548520 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:58.548520 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:58.548520 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:49:58.548520 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:49:58.548520 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:49:58.548520 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:49:58.548520 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:49:58.548520 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:49:58.548520 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 22:49:58.548520 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:58.553221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 470/945] 10.222.0.23 () {56 vars in 1117 bytes} [Mon May 9 22:49:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:58.554063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de0942-edf4-42c5-9f2f-2e38c732c882 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:58.632510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b22a04-8ce2-4089-a6ea-e4060d11787a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:58.669469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51cbeb8-9d60-4e1a-8ce3-510dab9133eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:58.669875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de0942-edf4-42c5-9f2f-2e38c732c882 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:58.670161 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cbeb8-9d60-4e1a-8ce3-510dab9133eb None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:58.670451 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cbeb8-9d60-4e1a-8ce3-510dab9133eb None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:58.670726 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cbeb8-9d60-4e1a-8ce3-510dab9133eb None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:58.671657 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51cbeb8-9d60-4e1a-8ce3-510dab9133eb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:58.674065 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51cbeb8-9d60-4e1a-8ce3-510dab9133eb None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:58.675534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: WARNING keystone.server.flask.common [None req-a51cbeb8-9d60-4e1a-8ce3-510dab9133eb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 09 22:49:58.700714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.sql.core [None req-a51cbeb8-9d60-4e1a-8ce3-510dab9133eb None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 09 22:49:58.700714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 09 22:49:58.700714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [parameters: {'id': '52147a5e73bc40689c1183718545d9af', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 09 22:49:58.700714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=79888) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 09 22:49:58.707540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718b642-b9d2-463c-b255-49935ba7994c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:58.708219 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718b642-b9d2-463c-b255-49935ba7994c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:58.709276 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718b642-b9d2-463c-b255-49935ba7994c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:58.709557 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718b642-b9d2-463c-b255-49935ba7994c None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:58.709998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-a51cbeb8-9d60-4e1a-8ce3-510dab9133eb None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 52147a5e73bc40689c1183718545d9af.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 52147a5e73bc40689c1183718545d9af. May 09 22:49:58.709998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:58.709998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 22:49:58.709998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 22:49:58.709998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 22:49:58.709998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 22:49:58.709998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 22:49:58.709998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application result = self._query(query) May 09 22:49:58.709998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 22:49:58.709998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application conn.query(q) May 09 22:49:58.709998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 22:49:58.709998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 22:49:58.709998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 22:49:58.710809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application result.read() May 09 22:49:58.710809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 22:49:58.710809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 22:49:58.710809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 22:49:58.710809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 22:49:58.710809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 22:49:58.710809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 22:49:58.710809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 22:49:58.710809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 22:49:58.710809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 09 22:49:58.710809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:58.710809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 09 22:49:58.710809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:58.710809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:58.710809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 09 22:49:58.710809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 09 22:49:58.710809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 09 22:49:58.710809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return project_ref.to_dict() May 09 22:49:58.711758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 22:49:58.711758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application next(self.gen) May 09 22:49:58.711758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 09 22:49:58.711758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application yield resource May 09 22:49:58.711758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 22:49:58.711758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application next(self.gen) May 09 22:49:58.711758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 09 22:49:58.711758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.session.rollback() May 09 22:49:58.711758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 09 22:49:58.711758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.force_reraise() May 09 22:49:58.711758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 09 22:49:58.711758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise self.value May 09 22:49:58.711758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 09 22:49:58.711758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 09 22:49:58.711758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 09 22:49:58.711758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application session.commit() May 09 22:49:58.711758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 09 22:49:58.712669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 09 22:49:58.712669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 09 22:49:58.712669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._prepare_impl() May 09 22:49:58.712669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 09 22:49:58.712669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.session.flush() May 09 22:49:58.712669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 09 22:49:58.712669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._flush(objects) May 09 22:49:58.712669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 09 22:49:58.712669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 09 22:49:58.712669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 09 22:49:58.712669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application compat.raise_( May 09 22:49:58.712669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 22:49:58.712669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception May 09 22:49:58.712669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 09 22:49:58.712669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application flush_context.execute() May 09 22:49:58.712669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 09 22:49:58.712669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rec.execute(self) May 09 22:49:58.713576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 09 22:49:58.713576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 09 22:49:58.713576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 09 22:49:58.713576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application _emit_insert_statements( May 09 22:49:58.713576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 09 22:49:58.713576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application c = connection._execute_20( May 09 22:49:58.713576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 09 22:49:58.713576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 09 22:49:58.713576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 09 22:49:58.713576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 09 22:49:58.713576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 09 22:49:58.713576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ret = self._execute_context( May 09 22:49:58.713576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 09 22:49:58.713576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 09 22:49:58.713576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 09 22:49:58.716189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 09 22:49:58.716189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 22:49:58.716189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception May 09 22:49:58.716189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 22:49:58.716189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 22:49:58.716189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 22:49:58.716189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 22:49:58.716189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 22:49:58.716189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application result = self._query(query) May 09 22:49:58.716189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 22:49:58.716189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application conn.query(q) May 09 22:49:58.716189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 22:49:58.716189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 22:49:58.716189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 22:49:58.716189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application result.read() May 09 22:49:58.716189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 22:49:58.716189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 22:49:58.717335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 22:49:58.717335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 22:49:58.717335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 22:49:58.717335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 22:49:58.717335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 22:49:58.717335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 22:49:58.717335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 09 22:49:58.717335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 09 22:49:58.717335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [parameters: {'id': '52147a5e73bc40689c1183718545d9af', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 09 22:49:58.717335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 09 22:49:58.717335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:58.717335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:49:58.717335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:58.717335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:58.717335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 09 22:49:58.718331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 09 22:49:58.718331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 09 22:49:58.718331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 09 22:49:58.718331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 09 22:49:58.718331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:58.718331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:49:58.718331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:58.718331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:58.718331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:58.718331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:58.718331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:58.718331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:58.718331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:58.718331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:58.718331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:58.718331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:58.718331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:58.720672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:58.720672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 09 22:49:58.720672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 09 22:49:58.720672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:58.720672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:58.720672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 09 22:49:58.720672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.Conflict( May 09 22:49:58.720672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 52147a5e73bc40689c1183718545d9af. May 09 22:49:58.720672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:58.721960 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718b642-b9d2-463c-b255-49935ba7994c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:58.721960 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718b642-b9d2-463c-b255-49935ba7994c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:58.722257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 476/946] 10.222.0.23 () {60 vars in 1140 bytes} [Mon May 9 22:49:58 2022] POST /identity/v3/projects => generated 262 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:58.722393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718b642-b9d2-463c-b255-49935ba7994c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:58.723256 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51cbeb8-9d60-4e1a-8ce3-510dab9133eb None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:58.739042 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 471/947] 10.222.0.23 () {56 vars in 1149 bytes} [Mon May 9 22:49:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:58.745645 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718b642-b9d2-463c-b255-49935ba7994c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:58.850418 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51cbeb8-9d60-4e1a-8ce3-510dab9133eb None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:58.860409 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718b642-b9d2-463c-b255-49935ba7994c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:58.885559 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53258560-bf9e-46eb-8431-6f2f07e019a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:58.886317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53258560-bf9e-46eb-8431-6f2f07e019a3 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:58.886596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53258560-bf9e-46eb-8431-6f2f07e019a3 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:58.886915 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53258560-bf9e-46eb-8431-6f2f07e019a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:58.894233 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53258560-bf9e-46eb-8431-6f2f07e019a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:58.895910 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53258560-bf9e-46eb-8431-6f2f07e019a3 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:58.903651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60197127-bf48-4651-8c8a-3c4e177663a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:58.904779 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60197127-bf48-4651-8c8a-3c4e177663a4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:58.905254 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60197127-bf48-4651-8c8a-3c4e177663a4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:58.905716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60197127-bf48-4651-8c8a-3c4e177663a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:58.911071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-53258560-bf9e-46eb-8431-6f2f07e019a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:58.911071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:58.911071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:58.911071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:58.911071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:58.911071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:58.911071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:58.911071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:58.911071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:58.911071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:58.911071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:58.911071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:58.911071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:58.911071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:58.911071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:58.911959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:58.911959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:58.911959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:58.911959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:58.911959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:58.911959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:58.911959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:58.911959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:58.911959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:58.911959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:58.911959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:58.911959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:58.911959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:58.911959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:58.911959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:58.911959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:58.911959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:58.913016 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:58.913016 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:58.913016 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:58.913016 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:58.913016 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:58.913016 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:58.913016 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:58.913016 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:58.913016 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 477/948] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:58.917999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53258560-bf9e-46eb-8431-6f2f07e019a3 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:58.926541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60197127-bf48-4651-8c8a-3c4e177663a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:58.928668 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60197127-bf48-4651-8c8a-3c4e177663a4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:58.953621 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-60197127-bf48-4651-8c8a-3c4e177663a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:58.953621 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:58.953621 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:58.953621 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:58.953621 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:58.953621 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:58.953621 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:58.953621 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:58.953621 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:58.953621 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:58.953621 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:58.953621 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:58.953621 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:49:58.953621 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:49:58.953621 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:49:58.954643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:49:58.954643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:58.954643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:58.954643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:58.954643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:58.954643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:58.954643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:58.954643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:58.954643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:49:58.954643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:58.954643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:58.954643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:58.954643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:58.954643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:58.954643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:58.954643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:58.954643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:58.955550 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:49:58.955550 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:49:58.955550 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:49:58.955550 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:49:58.955550 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 472/949] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:49:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:49:58.963402 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60197127-bf48-4651-8c8a-3c4e177663a4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:59.037941 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53258560-bf9e-46eb-8431-6f2f07e019a3 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:59.082404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79939a-3584-4b66-bf54-d485001cc2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:59.082404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79939a-3584-4b66-bf54-d485001cc2b3 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:59.082404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79939a-3584-4b66-bf54-d485001cc2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:59.082404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79939a-3584-4b66-bf54-d485001cc2b3 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:59.083591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79939a-3584-4b66-bf54-d485001cc2b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:59.084256 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79939a-3584-4b66-bf54-d485001cc2b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:59.086165 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79939a-3584-4b66-bf54-d485001cc2b3 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:59.089654 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60197127-bf48-4651-8c8a-3c4e177663a4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:59.112346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 478/950] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 9 22:49:58 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:49:59.125886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01eb4fa-cbee-414d-beef-859894632ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:59.126598 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01eb4fa-cbee-414d-beef-859894632ca3 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:59.126879 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01eb4fa-cbee-414d-beef-859894632ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:59.127154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01eb4fa-cbee-414d-beef-859894632ca3 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:59.127885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01eb4fa-cbee-414d-beef-859894632ca3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:59.128189 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01eb4fa-cbee-414d-beef-859894632ca3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:59.129610 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01eb4fa-cbee-414d-beef-859894632ca3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:59.147209 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 473/951] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:59.155518 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79939a-3584-4b66-bf54-d485001cc2b3 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:59.261266 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79939a-3584-4b66-bf54-d485001cc2b3 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:59.303587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083bf19-7e78-4a48-b271-eb269bd2d381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:59.304397 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083bf19-7e78-4a48-b271-eb269bd2d381 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:59.304679 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083bf19-7e78-4a48-b271-eb269bd2d381 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:59.304958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083bf19-7e78-4a48-b271-eb269bd2d381 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:59.312454 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0083bf19-7e78-4a48-b271-eb269bd2d381 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:59.313932 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0083bf19-7e78-4a48-b271-eb269bd2d381 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:59.329170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-0083bf19-7e78-4a48-b271-eb269bd2d381 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:59.329170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:49:59.329170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:49:59.329170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:49:59.329170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:49:59.329170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:49:59.329170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:49:59.329170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:49:59.329170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:49:59.329170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:49:59.329170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:49:59.329170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:49:59.329170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:49:59.329170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:49:59.329170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:49:59.330157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:49:59.330157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:49:59.330157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:49:59.330157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:49:59.330157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:49:59.330157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:49:59.330157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:49:59.330157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:49:59.330157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:49:59.330157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:49:59.330157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:49:59.330157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:49:59.330157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:49:59.330157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:49:59.330157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:49:59.330157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:49:59.330157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:49:59.331090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:49:59.331090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:49:59.331090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:49:59.331090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:49:59.331090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:49:59.331090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:49:59.331090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:49:59.331090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:49:59.331090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 479/952] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:49:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:49:59.338188 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01eb4fa-cbee-414d-beef-859894632ca3 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:59.438764 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01eb4fa-cbee-414d-beef-859894632ca3 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:59.473501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702f7eb-d357-4a13-a985-5d8647dd351b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:59.474495 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702f7eb-d357-4a13-a985-5d8647dd351b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:59.474984 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702f7eb-d357-4a13-a985-5d8647dd351b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:59.475446 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702f7eb-d357-4a13-a985-5d8647dd351b None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:59.476415 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702f7eb-d357-4a13-a985-5d8647dd351b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:49:59.476909 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702f7eb-d357-4a13-a985-5d8647dd351b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:59.478534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702f7eb-d357-4a13-a985-5d8647dd351b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:59.491881 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 474/953] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:49:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:49:59.497846 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083bf19-7e78-4a48-b271-eb269bd2d381 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:49:59.598457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083bf19-7e78-4a48-b271-eb269bd2d381 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:49:59.636181 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684e46f-e72f-4b8a-9344-4e0ee7585cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:49:59.636181 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684e46f-e72f-4b8a-9344-4e0ee7585cc0 None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:49:59.636181 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684e46f-e72f-4b8a-9344-4e0ee7585cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:49:59.636181 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684e46f-e72f-4b8a-9344-4e0ee7585cc0 None admin] PATH_INFO: `/v3/projects/0500e378ea374ca692e230bbd8781318/users/1a6f50dcebf146baae8a9f0b5b112795/roles/8b7c99010273443e88d3e81ce6924349` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:49:59.637819 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684e46f-e72f-4b8a-9344-4e0ee7585cc0 None admin] RBAC: Authorizing `identity:create_grant(project_id=0500e378ea374ca692e230bbd8781318, user_id=1a6f50dcebf146baae8a9f0b5b112795, role_id=8b7c99010273443e88d3e81ce6924349)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:49:59.642142 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684e46f-e72f-4b8a-9344-4e0ee7585cc0 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:49:59.656398 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 480/954] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:49:59 2022] PUT /identity/v3/projects/0500e378ea374ca692e230bbd8781318/users/1a6f50dcebf146baae8a9f0b5b112795/roles/8b7c99010273443e88d3e81ce6924349 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:50:00.646555 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e0952-76fa-444c-98e9-9a316a35db7f None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:00.647092 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e0952-76fa-444c-98e9-9a316a35db7f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:00.647364 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e0952-76fa-444c-98e9-9a316a35db7f None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:00.649251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 475/955] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:50:00.655569 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0a5f3-d47f-4463-a83f-6c2369cd2da7 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:00.656097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0a5f3-d47f-4463-a83f-6c2369cd2da7 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:00.656344 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0a5f3-d47f-4463-a83f-6c2369cd2da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:00.675373 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c1b0a5f3-d47f-4463-a83f-6c2369cd2da7 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:00.677028 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c1b0a5f3-d47f-4463-a83f-6c2369cd2da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:00.750221 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 481/956] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 22:50:00.766642 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e0952-76fa-444c-98e9-9a316a35db7f None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:00.825844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf63ef3-0fb0-41b4-98eb-f90d630e4747 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:00.826139 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf63ef3-0fb0-41b4-98eb-f90d630e4747 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:00.826417 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf63ef3-0fb0-41b4-98eb-f90d630e4747 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:00.827798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 482/957] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:50:00.833308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4420c-d6c7-4678-910d-d55f18a74fc4 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:00.833584 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4420c-d6c7-4678-910d-d55f18a74fc4 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:00.833853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4420c-d6c7-4678-910d-d55f18a74fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:00.854758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-1ac4420c-d6c7-4678-910d-d55f18a74fc4 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:00.856658 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1ac4420c-d6c7-4678-910d-d55f18a74fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:00.880395 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e0952-76fa-444c-98e9-9a316a35db7f None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:00.924361 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac148497-7829-4c7d-af6c-81014a88ad4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:00.925029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac148497-7829-4c7d-af6c-81014a88ad4e None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:00.925378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac148497-7829-4c7d-af6c-81014a88ad4e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:00.925573 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac148497-7829-4c7d-af6c-81014a88ad4e None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:00.926481 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac148497-7829-4c7d-af6c-81014a88ad4e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:00.927935 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac148497-7829-4c7d-af6c-81014a88ad4e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:00.928097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 483/958] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 22:50:00.950235 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 476/959] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 9 22:50:00 2022] POST /identity/v3/services => generated 242 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 22:50:00.951338 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4420c-d6c7-4678-910d-d55f18a74fc4 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:01.067984 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4420c-d6c7-4678-910d-d55f18a74fc4 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:01.113229 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149394b-a86f-4ce3-a1e6-7bc8ecdf5001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:01.114054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149394b-a86f-4ce3-a1e6-7bc8ecdf5001 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:01.114368 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149394b-a86f-4ce3-a1e6-7bc8ecdf5001 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:01.114697 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149394b-a86f-4ce3-a1e6-7bc8ecdf5001 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:01.115364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9149394b-a86f-4ce3-a1e6-7bc8ecdf5001 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:01.116809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9149394b-a86f-4ce3-a1e6-7bc8ecdf5001 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:01.130082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-9149394b-a86f-4ce3-a1e6-7bc8ecdf5001 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 22:50:01.130082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:01.130082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:50:01.130082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:50:01.130082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:50:01.130082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:50:01.130082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:50:01.130082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:50:01.130082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 22:50:01.130082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:01.130082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:50:01.130082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:01.130082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:01.130082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:01.130082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:01.130082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:01.130082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:01.131211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:01.131211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:01.131211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:01.131211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:01.131211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:01.131211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:01.131211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:50:01.131211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:50:01.131211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:50:01.131211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:50:01.131211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:01.131211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:01.131211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:01.131211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:01.131211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:01.131211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:01.132076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:01.132076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:50:01.132076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:01.132076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:01.132076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:01.132076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:01.132076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:01.132076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:01.132076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:01.132076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:01.132076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:50:01.132076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:50:01.132076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 22:50:01.132076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:01.132076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 484/960] 10.222.0.23 () {56 vars in 1126 bytes} [Mon May 9 22:50:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:50:01.139814 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac148497-7829-4c7d-af6c-81014a88ad4e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:01.239620 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac148497-7829-4c7d-af6c-81014a88ad4e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:01.273304 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbb520-7a35-4c06-a5ea-47f1e0d82f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:01.274023 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbb520-7a35-4c06-a5ea-47f1e0d82f7f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:01.274336 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbb520-7a35-4c06-a5ea-47f1e0d82f7f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:01.274613 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbb520-7a35-4c06-a5ea-47f1e0d82f7f None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:01.275313 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebbb520-7a35-4c06-a5ea-47f1e0d82f7f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:01.275615 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebbb520-7a35-4c06-a5ea-47f1e0d82f7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:01.276976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebbb520-7a35-4c06-a5ea-47f1e0d82f7f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:01.282034 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 477/961] 10.222.0.23 () {56 vars in 1122 bytes} [Mon May 9 22:50:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:01.288038 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149394b-a86f-4ce3-a1e6-7bc8ecdf5001 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:01.394477 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149394b-a86f-4ce3-a1e6-7bc8ecdf5001 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:01.428604 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3681883c-c750-4a69-9de3-e656fb0ef10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:01.429278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681883c-c750-4a69-9de3-e656fb0ef10c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:01.429558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681883c-c750-4a69-9de3-e656fb0ef10c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:01.429830 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681883c-c750-4a69-9de3-e656fb0ef10c None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:01.430559 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3681883c-c750-4a69-9de3-e656fb0ef10c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:01.430864 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3681883c-c750-4a69-9de3-e656fb0ef10c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:01.432221 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3681883c-c750-4a69-9de3-e656fb0ef10c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:01.437621 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 485/962] 10.222.0.23 () {56 vars in 1122 bytes} [Mon May 9 22:50:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:01.443825 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbb520-7a35-4c06-a5ea-47f1e0d82f7f None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:01.545639 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbb520-7a35-4c06-a5ea-47f1e0d82f7f None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:01.545868 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ebc78-3d28-4639-8440-a0b290eeef66 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:01.545963 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ebc78-3d28-4639-8440-a0b290eeef66 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:01.546209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ebc78-3d28-4639-8440-a0b290eeef66 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:01.549310 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 486/963] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:50:01.553105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27145a59-9c64-4ca3-a997-ee6352491789 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:01.553390 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27145a59-9c64-4ca3-a997-ee6352491789 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:01.553664 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27145a59-9c64-4ca3-a997-ee6352491789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:01.571952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-27145a59-9c64-4ca3-a997-ee6352491789 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:01.573110 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-27145a59-9c64-4ca3-a997-ee6352491789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:01.582775 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7c10b-33a1-4617-a734-f86f6f5bc3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:01.583515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7c10b-33a1-4617-a734-f86f6f5bc3f3 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:01.583808 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7c10b-33a1-4617-a734-f86f6f5bc3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:01.584173 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7c10b-33a1-4617-a734-f86f6f5bc3f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:01.585002 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7c10b-33a1-4617-a734-f86f6f5bc3f3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:01.586808 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7c10b-33a1-4617-a734-f86f6f5bc3f3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:01.608205 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 478/964] 10.222.0.23 () {60 vars in 1144 bytes} [Mon May 9 22:50:01 2022] POST /identity/v3/endpoints => generated 349 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:50:01.665168 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 487/965] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 22:50:01.684909 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7c10b-33a1-4617-a734-f86f6f5bc3f3 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:01.799070 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7c10b-33a1-4617-a734-f86f6f5bc3f3 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:01.834032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e9fc2-61e4-4aa0-9ced-c25d43433b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:01.834737 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e9fc2-61e4-4aa0-9ced-c25d43433b2f None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:01.835016 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e9fc2-61e4-4aa0-9ced-c25d43433b2f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:01.835289 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e9fc2-61e4-4aa0-9ced-c25d43433b2f None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:01.835999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e9fc2-61e4-4aa0-9ced-c25d43433b2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:01.837425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e9fc2-61e4-4aa0-9ced-c25d43433b2f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:01.838894 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: WARNING keystone.server.flask.common [None req-a82e9fc2-61e4-4aa0-9ced-c25d43433b2f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 09 22:50:01.853487 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.sql.core [None req-a82e9fc2-61e4-4aa0-9ced-c25d43433b2f None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 09 22:50:01.853487 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 09 22:50:01.853487 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [parameters: {'user_id': '2d4cd81985e243f29771298f8b1ac853', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 09 22:50:01.853487 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=79887) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 09 22:50:01.869912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-a82e9fc2-61e4-4aa0-9ced-c25d43433b2f None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 09 22:50:01.869912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:01.869912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 22:50:01.869912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 22:50:01.869912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 22:50:01.869912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 22:50:01.869912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 22:50:01.869912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application result = self._query(query) May 09 22:50:01.869912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 22:50:01.869912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application conn.query(q) May 09 22:50:01.869912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 22:50:01.869912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 22:50:01.869912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 22:50:01.869912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application result.read() May 09 22:50:01.869912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 22:50:01.870891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 22:50:01.870891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 22:50:01.870891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 22:50:01.870891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 22:50:01.870891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 22:50:01.870891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 22:50:01.870891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 22:50:01.870891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 09 22:50:01.870891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:01.870891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 09 22:50:01.870891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:01.870891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:01.870891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 09 22:50:01.870891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 09 22:50:01.870891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 09 22:50:01.870891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 09 22:50:01.870891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 22:50:01.870891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application next(self.gen) May 09 22:50:01.871843 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 09 22:50:01.871843 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application yield resource May 09 22:50:01.871843 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 22:50:01.871843 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application next(self.gen) May 09 22:50:01.871843 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 09 22:50:01.871843 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self.session.rollback() May 09 22:50:01.871843 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 09 22:50:01.871843 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self.force_reraise() May 09 22:50:01.871843 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 09 22:50:01.871843 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise self.value May 09 22:50:01.871843 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 09 22:50:01.871843 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 09 22:50:01.871843 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 09 22:50:01.871843 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application session.commit() May 09 22:50:01.871843 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 09 22:50:01.871843 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 09 22:50:01.871843 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 09 22:50:01.872743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._prepare_impl() May 09 22:50:01.872743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 09 22:50:01.872743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self.session.flush() May 09 22:50:01.872743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 09 22:50:01.872743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._flush(objects) May 09 22:50:01.872743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 09 22:50:01.872743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 09 22:50:01.872743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 09 22:50:01.872743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application compat.raise_( May 09 22:50:01.872743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 22:50:01.872743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception May 09 22:50:01.872743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 09 22:50:01.872743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application flush_context.execute() May 09 22:50:01.872743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 09 22:50:01.872743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rec.execute(self) May 09 22:50:01.872743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 09 22:50:01.872743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 09 22:50:01.874703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 09 22:50:01.874703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application _emit_insert_statements( May 09 22:50:01.874703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 09 22:50:01.874703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application result = connection._execute_20( May 09 22:50:01.874703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 09 22:50:01.874703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 09 22:50:01.874703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 09 22:50:01.874703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 09 22:50:01.874703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 09 22:50:01.874703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ret = self._execute_context( May 09 22:50:01.874703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 09 22:50:01.874703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 09 22:50:01.874703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 09 22:50:01.874703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 09 22:50:01.874703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 22:50:01.874703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception May 09 22:50:01.876719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 22:50:01.876719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 22:50:01.876719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 22:50:01.876719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 22:50:01.876719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 22:50:01.876719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application result = self._query(query) May 09 22:50:01.876719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 22:50:01.876719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application conn.query(q) May 09 22:50:01.876719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 22:50:01.876719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 22:50:01.876719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 22:50:01.876719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application result.read() May 09 22:50:01.876719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 22:50:01.876719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 22:50:01.876719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 22:50:01.876719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 22:50:01.882249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 22:50:01.882249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 22:50:01.882249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 22:50:01.882249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 22:50:01.882249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 09 22:50:01.882249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 09 22:50:01.882249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [parameters: {'user_id': '2d4cd81985e243f29771298f8b1ac853', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 09 22:50:01.882249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 09 22:50:01.882249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:01.882249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:50:01.882249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:01.882249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:01.882249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:01.882249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:01.882249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:01.883626 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:01.883626 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:01.883626 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:01.883626 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:01.883626 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:01.883626 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:01.883626 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:01.883626 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 09 22:50:01.883626 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 09 22:50:01.883626 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:01.883626 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:01.883626 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:50:01.883626 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:50:01.883626 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:50:01.883626 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:50:01.883626 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 09 22:50:01.883626 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 09 22:50:01.885846 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 09 22:50:01.885846 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 09 22:50:01.885846 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 09 22:50:01.885846 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 09 22:50:01.885846 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 09 22:50:01.885846 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:01.885846 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 479/966] 10.222.0.23 () {60 vars in 1131 bytes} [Mon May 9 22:50:01 2022] POST /identity/v3/users => generated 160 bytes in 187 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 09 22:50:01.886727 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27145a59-9c64-4ca3-a997-ee6352491789 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:01.960191 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c6ea6-b6d4-4d14-9bbe-6f12b073d304 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:01.960467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c6ea6-b6d4-4d14-9bbe-6f12b073d304 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:01.960655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c6ea6-b6d4-4d14-9bbe-6f12b073d304 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:01.966765 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 480/967] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:50:01.971986 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296544b5-408b-4a0e-9172-21d043e5dacd None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:01.972269 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296544b5-408b-4a0e-9172-21d043e5dacd None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:01.973104 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296544b5-408b-4a0e-9172-21d043e5dacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:01.981844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27145a59-9c64-4ca3-a997-ee6352491789 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:01.991929 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-296544b5-408b-4a0e-9172-21d043e5dacd None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:01.994074 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-296544b5-408b-4a0e-9172-21d043e5dacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:02.018722 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838bcc8-1896-41d2-9844-74607fb14e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:02.020381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838bcc8-1896-41d2-9844-74607fb14e12 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:02.020665 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838bcc8-1896-41d2-9844-74607fb14e12 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:02.021098 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838bcc8-1896-41d2-9844-74607fb14e12 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:02.030565 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838bcc8-1896-41d2-9844-74607fb14e12 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:02.032175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838bcc8-1896-41d2-9844-74607fb14e12 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:02.052797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-3838bcc8-1896-41d2-9844-74607fb14e12 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 22:50:02.052797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:02.052797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:02.052797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:02.052797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:02.052797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:02.052797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:02.052797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:02.052797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:02.052797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:02.052797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:02.052797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:02.052797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:50:02.052797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:50:02.052797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:50:02.052797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:50:02.053981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:02.053981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:02.053981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:50:02.053981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:50:02.053981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:50:02.053981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:50:02.053981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:02.053981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:02.053981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:02.053981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:02.053981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:02.053981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:50:02.053981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:02.053981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:02.053981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:02.053981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:02.053981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:02.053981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:02.055166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:02.055166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:02.055166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:50:02.055166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:50:02.055166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:50:02.055166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:50:02.055166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:50:02.055166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:50:02.055166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 22:50:02.055166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:02.055166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 488/968] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:50:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:50:02.060998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838bcc8-1896-41d2-9844-74607fb14e12 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:02.078203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 481/969] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 22:50:02.098893 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296544b5-408b-4a0e-9172-21d043e5dacd None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:02.183730 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838bcc8-1896-41d2-9844-74607fb14e12 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:02.220869 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296544b5-408b-4a0e-9172-21d043e5dacd None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:02.228507 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b874f4-f6f1-4036-a49e-92d0ff0fb357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:02.229189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b874f4-f6f1-4036-a49e-92d0ff0fb357 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:02.229472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b874f4-f6f1-4036-a49e-92d0ff0fb357 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:02.229746 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b874f4-f6f1-4036-a49e-92d0ff0fb357 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:02.231010 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b874f4-f6f1-4036-a49e-92d0ff0fb357 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:02.231317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b874f4-f6f1-4036-a49e-92d0ff0fb357 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:02.233106 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b874f4-f6f1-4036-a49e-92d0ff0fb357 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:02.261117 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ee480-f287-4b59-ab73-9056b62ea932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:02.262861 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ee480-f287-4b59-ab73-9056b62ea932 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:02.263145 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ee480-f287-4b59-ab73-9056b62ea932 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:02.263423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ee480-f287-4b59-ab73-9056b62ea932 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:02.276924 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 489/970] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 9 22:50:02 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:02.283386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ee480-f287-4b59-ab73-9056b62ea932 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:50:02.292883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-1c0ee480-f287-4b59-ab73-9056b62ea932 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 22:50:02.292883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:02.292883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:02.292883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:02.292883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:02.292883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:02.292883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:02.292883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:02.292883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:02.292883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:02.292883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:02.292883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:02.292883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:50:02.292883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:50:02.292883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:50:02.292883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:50:02.294300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:50:02.294300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:50:02.294300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:50:02.294300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:50:02.294300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:02.294300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:02.294300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:02.294300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:02.294300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:02.294300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:02.294300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:02.294300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:50:02.294300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:02.294300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:02.294300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:02.294300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:02.295416 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:02.295416 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:02.295416 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:02.295416 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:02.295416 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:50:02.295416 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:50:02.295416 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:50:02.295416 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:50:02.295416 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:50:02.295416 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:50:02.295416 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 22:50:02.295416 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:02.302163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 482/971] 10.222.0.23 () {56 vars in 1120 bytes} [Mon May 9 22:50:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:50:02.303788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b874f4-f6f1-4036-a49e-92d0ff0fb357 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:02.414062 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b874f4-f6f1-4036-a49e-92d0ff0fb357 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:02.450097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc57958-f9e9-4cae-9a8a-582bc0d83958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:02.450797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc57958-f9e9-4cae-9a8a-582bc0d83958 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:02.451075 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc57958-f9e9-4cae-9a8a-582bc0d83958 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:02.451345 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc57958-f9e9-4cae-9a8a-582bc0d83958 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:02.452095 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc57958-f9e9-4cae-9a8a-582bc0d83958 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:02.452397 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc57958-f9e9-4cae-9a8a-582bc0d83958 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:02.453795 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc57958-f9e9-4cae-9a8a-582bc0d83958 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:02.463952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 490/972] 10.222.0.23 () {56 vars in 1115 bytes} [Mon May 9 22:50:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:02.469568 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ee480-f287-4b59-ab73-9056b62ea932 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:02.572562 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ee480-f287-4b59-ab73-9056b62ea932 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:02.605895 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911ca05-29d7-44db-94b8-7a5985cc7752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:02.606800 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911ca05-29d7-44db-94b8-7a5985cc7752 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:02.607137 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911ca05-29d7-44db-94b8-7a5985cc7752 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:02.607468 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911ca05-29d7-44db-94b8-7a5985cc7752 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:02.619632 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911ca05-29d7-44db-94b8-7a5985cc7752 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:02.624417 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911ca05-29d7-44db-94b8-7a5985cc7752 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:02.645355 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-5911ca05-29d7-44db-94b8-7a5985cc7752 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:50:02.645355 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:02.645355 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:02.645355 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:02.645355 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:02.645355 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:02.645355 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:02.645355 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:02.645355 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:02.645355 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:02.645355 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:02.645355 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:02.645355 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:50:02.645355 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:50:02.645355 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:50:02.646655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:50:02.646655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:02.646655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:02.646655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:02.646655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:02.646655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:02.646655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:02.646655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:02.646655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:50:02.646655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:02.646655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:02.646655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:02.646655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:02.646655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:02.646655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:02.646655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:02.646655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:02.647774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:50:02.647774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:50:02.647774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:50:02.647774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:02.648201 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 483/973] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:50:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:50:02.652180 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc57958-f9e9-4cae-9a8a-582bc0d83958 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:02.747042 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc57958-f9e9-4cae-9a8a-582bc0d83958 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:02.778583 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1537c5-a070-43bc-a645-957db6c39653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:02.779240 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1537c5-a070-43bc-a645-957db6c39653 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:02.779517 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1537c5-a070-43bc-a645-957db6c39653 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:02.779785 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1537c5-a070-43bc-a645-957db6c39653 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:02.780492 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1537c5-a070-43bc-a645-957db6c39653 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:02.780794 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1537c5-a070-43bc-a645-957db6c39653 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:02.782172 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1537c5-a070-43bc-a645-957db6c39653 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:02.798000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 491/974] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:02.799215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911ca05-29d7-44db-94b8-7a5985cc7752 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:02.899290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911ca05-29d7-44db-94b8-7a5985cc7752 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:02.931335 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49203cd-fd61-4ba2-8052-80ad4d6a1ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:02.932147 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49203cd-fd61-4ba2-8052-80ad4d6a1ceb None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:02.932481 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49203cd-fd61-4ba2-8052-80ad4d6a1ceb None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:02.932755 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49203cd-fd61-4ba2-8052-80ad4d6a1ceb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:02.946193 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49203cd-fd61-4ba2-8052-80ad4d6a1ceb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:02.947640 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49203cd-fd61-4ba2-8052-80ad4d6a1ceb None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:02.962252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-a49203cd-fd61-4ba2-8052-80ad4d6a1ceb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 22:50:02.962252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:02.962252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:02.962252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:02.962252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:02.962252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:02.962252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:02.962252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:02.962252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:02.962252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:02.962252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:02.962252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:02.962252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:50:02.962252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:50:02.962252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:50:02.962252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:50:02.963308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:02.963308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:02.963308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:50:02.963308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:50:02.963308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:50:02.963308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:50:02.963308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:02.963308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:02.963308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:02.963308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:02.963308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:02.963308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:50:02.963308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:02.963308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:02.963308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:02.963308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:02.963308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:02.963308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:02.964286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:02.964286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:02.964286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:50:02.964286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:50:02.964286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:50:02.964286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:50:02.964286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:50:02.964286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:50:02.964286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 22:50:02.964286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:02.966580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 484/975] 10.222.0.23 () {56 vars in 1117 bytes} [Mon May 9 22:50:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:50:02.973261 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1537c5-a070-43bc-a645-957db6c39653 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:03.069701 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1537c5-a070-43bc-a645-957db6c39653 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:03.102237 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5afa9-0b3b-4c70-b1e6-ea7f2a881061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:03.102903 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5afa9-0b3b-4c70-b1e6-ea7f2a881061 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:03.103185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5afa9-0b3b-4c70-b1e6-ea7f2a881061 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:03.103512 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5afa9-0b3b-4c70-b1e6-ea7f2a881061 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:03.104343 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b5afa9-0b3b-4c70-b1e6-ea7f2a881061 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:03.104649 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b5afa9-0b3b-4c70-b1e6-ea7f2a881061 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:03.106068 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b5afa9-0b3b-4c70-b1e6-ea7f2a881061 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:03.129051 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 492/976] 10.222.0.23 () {56 vars in 1149 bytes} [Mon May 9 22:50:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:03.130461 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49203cd-fd61-4ba2-8052-80ad4d6a1ceb None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:03.228860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49203cd-fd61-4ba2-8052-80ad4d6a1ceb None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:03.265024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29179d7a-af8e-4f29-9c6c-07fd1d072ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:03.265674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29179d7a-af8e-4f29-9c6c-07fd1d072ecb None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:03.265955 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29179d7a-af8e-4f29-9c6c-07fd1d072ecb None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:03.266272 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29179d7a-af8e-4f29-9c6c-07fd1d072ecb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:03.330659 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29179d7a-af8e-4f29-9c6c-07fd1d072ecb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:03.332139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29179d7a-af8e-4f29-9c6c-07fd1d072ecb None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:03.336007 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38eae09-526e-4cb5-a27b-15ee451b8ec4 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:03.336299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38eae09-526e-4cb5-a27b-15ee451b8ec4 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:03.336576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38eae09-526e-4cb5-a27b-15ee451b8ec4 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:03.338100 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 493/977] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:50:03.342847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a3109-1e0a-444f-a575-b396fb0611eb None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:03.343142 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a3109-1e0a-444f-a575-b396fb0611eb None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:03.343418 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a3109-1e0a-444f-a575-b396fb0611eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:03.360557 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5a0a3109-1e0a-444f-a575-b396fb0611eb None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:03.361733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5a0a3109-1e0a-444f-a575-b396fb0611eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:03.434231 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 485/978] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:50:03 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 22:50:03.435325 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 494/979] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 22:50:03.443294 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29179d7a-af8e-4f29-9c6c-07fd1d072ecb None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:03.458000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a3109-1e0a-444f-a575-b396fb0611eb None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:03.540574 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29179d7a-af8e-4f29-9c6c-07fd1d072ecb None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:03.572020 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a3109-1e0a-444f-a575-b396fb0611eb None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:03.577475 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224f1f1-8e49-48f2-aae3-e2c5acc119e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:03.578298 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224f1f1-8e49-48f2-aae3-e2c5acc119e0 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:03.578579 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224f1f1-8e49-48f2-aae3-e2c5acc119e0 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:03.578934 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224f1f1-8e49-48f2-aae3-e2c5acc119e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:03.589384 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224f1f1-8e49-48f2-aae3-e2c5acc119e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:03.590856 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224f1f1-8e49-48f2-aae3-e2c5acc119e0 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:03.608535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-6224f1f1-8e49-48f2-aae3-e2c5acc119e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:50:03.608535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:03.608535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:03.608535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:03.608535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:03.608535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:03.608535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:03.608535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:03.608535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:03.608535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:03.608535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:03.608535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:03.608535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:50:03.608535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:50:03.608535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:50:03.609782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:50:03.609782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:03.609782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:03.609782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:03.609782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:03.609782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:03.609782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:03.609782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:03.609782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:50:03.609782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:03.609782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:03.609782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:03.609782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:03.609782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:03.609782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:03.609782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:03.609782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:03.610721 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:50:03.610721 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:50:03.610721 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:50:03.610721 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:03.614581 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 486/980] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:50:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:50:03.615511 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224f1f1-8e49-48f2-aae3-e2c5acc119e0 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:03.628005 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae17d6d-3108-4057-877b-049ec2992d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:03.628919 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae17d6d-3108-4057-877b-049ec2992d37 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:03.629266 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae17d6d-3108-4057-877b-049ec2992d37 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:03.629607 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae17d6d-3108-4057-877b-049ec2992d37 None admin] PATH_INFO: `/v3/services/compute` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:03.630359 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae17d6d-3108-4057-877b-049ec2992d37 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:03.631738 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae17d6d-3108-4057-877b-049ec2992d37 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:03.648057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-5ae17d6d-3108-4057-877b-049ec2992d37 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 09 22:50:03.648057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:03.648057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:50:03.648057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:50:03.648057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:50:03.648057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:50:03.648057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:50:03.648057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:50:03.648057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 22:50:03.648057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:03.648057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:50:03.648057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:03.648057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:03.648057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:03.648057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:03.648057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:03.648057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:03.649153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:03.649153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:03.649153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:03.649153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:03.649153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:03.649153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:03.649153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:50:03.649153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:50:03.649153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:50:03.649153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:50:03.649153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:03.649153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:03.649153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:03.649153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:03.649153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:03.649153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:03.650026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:03.650026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:50:03.650026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:03.650026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:03.650026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:03.650026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:03.650026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:03.650026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:03.650026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:03.650026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:03.650026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:50:03.650026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:50:03.650026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 22:50:03.650026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:03.654227 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 495/981] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:50:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:50:03.655359 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae17d6d-3108-4057-877b-049ec2992d37 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:03.717879 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224f1f1-8e49-48f2-aae3-e2c5acc119e0 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:03.750939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae17d6d-3108-4057-877b-049ec2992d37 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:03.751483 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412d711-c962-461f-bcd8-9367caaf7ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:03.751756 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412d711-c962-461f-bcd8-9367caaf7ce1 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:03.751939 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412d711-c962-461f-bcd8-9367caaf7ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:03.752182 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412d711-c962-461f-bcd8-9367caaf7ce1 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:03.752887 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412d711-c962-461f-bcd8-9367caaf7ce1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:03.753184 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412d711-c962-461f-bcd8-9367caaf7ce1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:03.754616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412d711-c962-461f-bcd8-9367caaf7ce1 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:03.771127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 487/982] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:03.772386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412d711-c962-461f-bcd8-9367caaf7ce1 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:03.787094 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca4334-564a-4a30-b38e-04fcbbbde363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:03.787779 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca4334-564a-4a30-b38e-04fcbbbde363 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:03.788062 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca4334-564a-4a30-b38e-04fcbbbde363 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:03.788341 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca4334-564a-4a30-b38e-04fcbbbde363 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:03.789056 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca4334-564a-4a30-b38e-04fcbbbde363 None admin] RBAC: Adding query filter params (name=compute) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:03.789363 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca4334-564a-4a30-b38e-04fcbbbde363 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:03.790887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca4334-564a-4a30-b38e-04fcbbbde363 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:03.795957 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 496/983] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 9 22:50:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:03.801710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca4334-564a-4a30-b38e-04fcbbbde363 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:03.875710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412d711-c962-461f-bcd8-9367caaf7ce1 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:03.899784 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca4334-564a-4a30-b38e-04fcbbbde363 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:03.908523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f7b4f-f0f7-4441-99c0-e82b93f4d3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:03.909319 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f7b4f-f0f7-4441-99c0-e82b93f4d3da None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:03.909624 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f7b4f-f0f7-4441-99c0-e82b93f4d3da None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:03.909921 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f7b4f-f0f7-4441-99c0-e82b93f4d3da None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:03.919236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f7b4f-f0f7-4441-99c0-e82b93f4d3da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:03.920699 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f7b4f-f0f7-4441-99c0-e82b93f4d3da None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:03.937636 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1dd788-a43f-4f0e-9366-c9c995280361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:03.938140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-4d7f7b4f-f0f7-4441-99c0-e82b93f4d3da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:50:03.938140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:03.938140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:03.938140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:03.938140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:03.938140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:03.938140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:03.938140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:03.938140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:03.938140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:03.938140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:03.938140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:03.938140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:50:03.938140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:50:03.938140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:50:03.939041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:50:03.939041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:03.939041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:03.939041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:03.939041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:03.939041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:03.939041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:03.939041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:03.939041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:50:03.939041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:03.939041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:03.939041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:03.939041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:03.939041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:03.939041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:03.939041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:03.939041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:03.945211 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:50:03.945211 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:50:03.945211 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:50:03.945211 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:50:03.945211 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:50:03.945211 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:50:03.945211 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:50:03.945211 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:03.945657 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1dd788-a43f-4f0e-9366-c9c995280361 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:03.945657 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1dd788-a43f-4f0e-9366-c9c995280361 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:03.945657 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1dd788-a43f-4f0e-9366-c9c995280361 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:03.945657 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1dd788-a43f-4f0e-9366-c9c995280361 None admin] RBAC: Adding query filter params (type=compute) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:03.945981 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 488/984] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:50:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:50:03.946055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1dd788-a43f-4f0e-9366-c9c995280361 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:03.946055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1dd788-a43f-4f0e-9366-c9c995280361 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:03.946222 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f7b4f-f0f7-4441-99c0-e82b93f4d3da None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:03.947361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 497/985] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 9 22:50:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:03.953204 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1dd788-a43f-4f0e-9366-c9c995280361 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:04.040725 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f7b4f-f0f7-4441-99c0-e82b93f4d3da None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:04.051139 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1dd788-a43f-4f0e-9366-c9c995280361 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:04.072235 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d42076-376a-420b-8d6f-1515a701345b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:04.072884 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d42076-376a-420b-8d6f-1515a701345b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:04.073162 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d42076-376a-420b-8d6f-1515a701345b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:04.073435 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d42076-376a-420b-8d6f-1515a701345b None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:04.074231 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d42076-376a-420b-8d6f-1515a701345b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:04.074537 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d42076-376a-420b-8d6f-1515a701345b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:04.075954 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d42076-376a-420b-8d6f-1515a701345b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:04.083499 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99b46c-673b-46bc-b33e-77ffee71d906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:04.084144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99b46c-673b-46bc-b33e-77ffee71d906 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:04.084432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99b46c-673b-46bc-b33e-77ffee71d906 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:04.084708 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99b46c-673b-46bc-b33e-77ffee71d906 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:04.085482 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99b46c-673b-46bc-b33e-77ffee71d906 None admin] RBAC: Adding query filter params (interface=public, service_id=fe44505779094070bf417601ea46beb2, region_id=RegionOne) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:04.085795 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99b46c-673b-46bc-b33e-77ffee71d906 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:04.087166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99b46c-673b-46bc-b33e-77ffee71d906 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:04.087380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 489/986] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:50:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:04.088671 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7efbef-a3c3-4990-ad46-7bc5ebac1677 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:04.088960 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7efbef-a3c3-4990-ad46-7bc5ebac1677 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:04.089244 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7efbef-a3c3-4990-ad46-7bc5ebac1677 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:04.090729 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 490/987] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:50:04.093105 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7efbef-a3c3-4990-ad46-7bc5ebac1677 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:04.095807 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 498/988] 10.222.0.23 () {56 vars in 1251 bytes} [Mon May 9 22:50:03 2022] GET /identity/v3/endpoints?service_id=fe44505779094070bf417601ea46beb2&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:04.097184 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebb8c7-85d5-47e5-a444-4fb282740154 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:04.097507 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebb8c7-85d5-47e5-a444-4fb282740154 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:04.097842 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebb8c7-85d5-47e5-a444-4fb282740154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:04.113941 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-aaebb8c7-85d5-47e5-a444-4fb282740154 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:04.115231 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-aaebb8c7-85d5-47e5-a444-4fb282740154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:04.178429 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 499/989] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:04 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:50:04.179237 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebb8c7-85d5-47e5-a444-4fb282740154 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:04.189650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7efbef-a3c3-4990-ad46-7bc5ebac1677 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:04.224586 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f245572-c4a3-4f20-ae0d-91965cf0c9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:04.225336 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f245572-c4a3-4f20-ae0d-91965cf0c9b8 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:04.225668 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f245572-c4a3-4f20-ae0d-91965cf0c9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:04.226005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f245572-c4a3-4f20-ae0d-91965cf0c9b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:04.226908 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f245572-c4a3-4f20-ae0d-91965cf0c9b8 None admin] RBAC: Adding query filter params (role.id=8b7c99010273443e88d3e81ce6924349, scope.project.id=0500e378ea374ca692e230bbd8781318, user.id=1a6f50dcebf146baae8a9f0b5b112795) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:04.227317 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f245572-c4a3-4f20-ae0d-91965cf0c9b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:04.228735 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f245572-c4a3-4f20-ae0d-91965cf0c9b8 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:04.234576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 491/990] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:50:04 2022] GET /identity/v3/role_assignments?user.id=1a6f50dcebf146baae8a9f0b5b112795&scope.project.id=0500e378ea374ca692e230bbd8781318&role.id=8b7c99010273443e88d3e81ce6924349 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:04.235329 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f245572-c4a3-4f20-ae0d-91965cf0c9b8 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:04.280011 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebb8c7-85d5-47e5-a444-4fb282740154 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:04.315630 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad909d1-5344-413a-95b3-07278f15e9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:04.317124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad909d1-5344-413a-95b3-07278f15e9b1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:04.317413 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad909d1-5344-413a-95b3-07278f15e9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:04.317690 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad909d1-5344-413a-95b3-07278f15e9b1 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:04.318416 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad909d1-5344-413a-95b3-07278f15e9b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:04.319816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad909d1-5344-413a-95b3-07278f15e9b1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:04.326434 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 500/991] 10.222.0.23 () {56 vars in 1087 bytes} [Mon May 9 22:50:04 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 22:50:04.353945 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f245572-c4a3-4f20-ae0d-91965cf0c9b8 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:04.393149 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c531484-5c90-4b85-b95b-ac7f50586637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:04.394019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c531484-5c90-4b85-b95b-ac7f50586637 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:04.394378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c531484-5c90-4b85-b95b-ac7f50586637 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:04.394655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c531484-5c90-4b85-b95b-ac7f50586637 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:04.395336 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c531484-5c90-4b85-b95b-ac7f50586637 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:04.396785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c531484-5c90-4b85-b95b-ac7f50586637 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:04.409875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-9c531484-5c90-4b85-b95b-ac7f50586637 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 22:50:04.409875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:04.409875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:50:04.409875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:50:04.409875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:50:04.409875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:50:04.409875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:50:04.409875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:50:04.409875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 22:50:04.409875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:04.409875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:50:04.409875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:04.409875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:04.409875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:04.409875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:04.409875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:04.409875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:04.411237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:04.411237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:04.411237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:04.411237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:04.411237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:04.411237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:04.411237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:50:04.411237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:50:04.411237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:50:04.411237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:50:04.411237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:04.411237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:04.411237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:04.411237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:04.411237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:04.411237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:04.412098 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:04.412098 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:50:04.412098 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:04.412098 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:04.412098 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:04.412098 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:04.412098 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:04.412098 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:04.412098 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:04.412098 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:04.412098 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:50:04.412098 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:50:04.412098 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 22:50:04.412098 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:04.413568 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 492/992] 10.222.0.23 () {56 vars in 1126 bytes} [Mon May 9 22:50:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:50:04.417698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad909d1-5344-413a-95b3-07278f15e9b1 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:04.524793 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad909d1-5344-413a-95b3-07278f15e9b1 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:04.558656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394dbb3-e979-4033-9b55-8cbb1ec8a539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:04.559379 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394dbb3-e979-4033-9b55-8cbb1ec8a539 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:04.559662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394dbb3-e979-4033-9b55-8cbb1ec8a539 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:04.559918 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394dbb3-e979-4033-9b55-8cbb1ec8a539 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:04.560669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a394dbb3-e979-4033-9b55-8cbb1ec8a539 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:04.560973 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a394dbb3-e979-4033-9b55-8cbb1ec8a539 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:04.562359 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a394dbb3-e979-4033-9b55-8cbb1ec8a539 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:04.568028 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 501/993] 10.222.0.23 () {56 vars in 1122 bytes} [Mon May 9 22:50:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:04.573878 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c531484-5c90-4b85-b95b-ac7f50586637 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:04.679845 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c531484-5c90-4b85-b95b-ac7f50586637 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:04.712498 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37b6d4-b4a4-4870-930e-5c7a60fab2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:04.713152 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37b6d4-b4a4-4870-930e-5c7a60fab2e4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:04.713429 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37b6d4-b4a4-4870-930e-5c7a60fab2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:04.713698 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37b6d4-b4a4-4870-930e-5c7a60fab2e4 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:04.714414 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37b6d4-b4a4-4870-930e-5c7a60fab2e4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:04.714715 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37b6d4-b4a4-4870-930e-5c7a60fab2e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:04.716061 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37b6d4-b4a4-4870-930e-5c7a60fab2e4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:04.725669 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 493/994] 10.222.0.23 () {56 vars in 1122 bytes} [Mon May 9 22:50:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:04.726745 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394dbb3-e979-4033-9b55-8cbb1ec8a539 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:04.821966 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394dbb3-e979-4033-9b55-8cbb1ec8a539 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:04.855408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5290af-bf9b-488f-a8d9-ac41ad3c44f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:04.856079 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5290af-bf9b-488f-a8d9-ac41ad3c44f9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:04.856361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5290af-bf9b-488f-a8d9-ac41ad3c44f9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:04.856636 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5290af-bf9b-488f-a8d9-ac41ad3c44f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:04.857438 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5290af-bf9b-488f-a8d9-ac41ad3c44f9 None admin] RBAC: Adding query filter params (interface=admin, service_id=31572c3ef7d948e58386d2a29f51f807, region_id=RegionOne) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:04.857750 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5290af-bf9b-488f-a8d9-ac41ad3c44f9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:04.859423 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5290af-bf9b-488f-a8d9-ac41ad3c44f9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:04.866258 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 502/995] 10.222.0.23 () {56 vars in 1249 bytes} [Mon May 9 22:50:04 2022] GET /identity/v3/endpoints?service_id=31572c3ef7d948e58386d2a29f51f807&interface=admin®ion_id=RegionOne => generated 193 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:04.870518 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37b6d4-b4a4-4870-930e-5c7a60fab2e4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:04.969252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37b6d4-b4a4-4870-930e-5c7a60fab2e4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:05.002645 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0c3c5-5cd3-4090-a281-0ab4345cf177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:05.003306 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0c3c5-5cd3-4090-a281-0ab4345cf177 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:05.003652 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0c3c5-5cd3-4090-a281-0ab4345cf177 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:05.004422 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0c3c5-5cd3-4090-a281-0ab4345cf177 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:05.005090 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0c3c5-5cd3-4090-a281-0ab4345cf177 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:05.006472 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0c3c5-5cd3-4090-a281-0ab4345cf177 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:05.014034 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 494/996] 10.222.0.23 () {56 vars in 1087 bytes} [Mon May 9 22:50:04 2022] GET /identity/v3/services => generated 1727 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 22:50:06.501503 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a878bd7-9e21-4cff-a888-d37b6d630c78 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:06.502026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a878bd7-9e21-4cff-a888-d37b6d630c78 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:06.502026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a878bd7-9e21-4cff-a888-d37b6d630c78 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:06.506174 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 503/997] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:50:06.509236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03679c6-e861-45ef-a883-514a389d386d None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:06.509635 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03679c6-e861-45ef-a883-514a389d386d None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:06.509799 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03679c6-e861-45ef-a883-514a389d386d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:06.525974 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b03679c6-e861-45ef-a883-514a389d386d None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:06.527171 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b03679c6-e861-45ef-a883-514a389d386d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:06.569725 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17d705-6dd3-4562-94e6-56c393c644c0 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:06.570014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17d705-6dd3-4562-94e6-56c393c644c0 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:06.570319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17d705-6dd3-4562-94e6-56c393c644c0 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:06.571808 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 504/998] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:50:06.576629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98758307-4eae-48dd-8658-2dc41bc278a5 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:06.576913 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98758307-4eae-48dd-8658-2dc41bc278a5 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:06.577190 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98758307-4eae-48dd-8658-2dc41bc278a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:06.592436 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 495/999] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:50:06.594204 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-98758307-4eae-48dd-8658-2dc41bc278a5 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:06.595373 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-98758307-4eae-48dd-8658-2dc41bc278a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:06.614602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03679c6-e861-45ef-a883-514a389d386d None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:06.666579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 505/1000] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 22:50:06.686994 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98758307-4eae-48dd-8658-2dc41bc278a5 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:06.716532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03679c6-e861-45ef-a883-514a389d386d None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:06.751250 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bec125-11f0-4315-bf41-e908535ca671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:06.752134 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bec125-11f0-4315-bf41-e908535ca671 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:06.752314 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bec125-11f0-4315-bf41-e908535ca671 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:06.752560 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bec125-11f0-4315-bf41-e908535ca671 None admin] PATH_INFO: `/v3/services/image` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:06.753296 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bec125-11f0-4315-bf41-e908535ca671 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:06.755171 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bec125-11f0-4315-bf41-e908535ca671 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:06.766450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-79bec125-11f0-4315-bf41-e908535ca671 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 09 22:50:06.766450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:06.766450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:50:06.766450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:50:06.766450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:50:06.766450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:50:06.766450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:50:06.766450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:50:06.766450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 22:50:06.766450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:06.766450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:50:06.766450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:06.766450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:06.766450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:06.766450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:06.766450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:06.766450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:06.767526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:06.767526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:06.767526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:06.767526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:06.767526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:06.767526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:06.767526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:50:06.767526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:50:06.767526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:50:06.767526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:50:06.767526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:06.767526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:06.767526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:06.767526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:06.767526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:06.767526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:06.768521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:06.768521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:50:06.768521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:06.768521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:06.768521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:06.768521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:06.768521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:06.768521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:06.768521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:06.768521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:06.768521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:50:06.768521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:50:06.768521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 22:50:06.768521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:06.768521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 496/1001] 10.222.0.23 () {56 vars in 1105 bytes} [Mon May 9 22:50:06 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:50:06.773428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bec125-11f0-4315-bf41-e908535ca671 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:06.788748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98758307-4eae-48dd-8658-2dc41bc278a5 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:06.822315 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff4268-79f0-4478-84ba-c195fe4efd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:06.823096 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff4268-79f0-4478-84ba-c195fe4efd9f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:06.823371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff4268-79f0-4478-84ba-c195fe4efd9f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:06.823669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff4268-79f0-4478-84ba-c195fe4efd9f None admin] PATH_INFO: `/v3/services/compute` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:06.824298 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff4268-79f0-4478-84ba-c195fe4efd9f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:06.825646 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff4268-79f0-4478-84ba-c195fe4efd9f None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:06.837124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-ceff4268-79f0-4478-84ba-c195fe4efd9f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 09 22:50:06.837124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:06.837124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:50:06.837124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:50:06.837124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:50:06.837124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:50:06.837124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:50:06.837124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:50:06.837124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 22:50:06.837124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:06.837124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:50:06.837124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:06.837124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:06.837124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:06.837124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:06.837124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:06.837124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:06.838194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:06.838194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:06.838194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:06.838194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:06.838194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:06.838194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:06.838194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:50:06.838194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:50:06.838194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:50:06.838194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:50:06.838194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:06.838194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:06.838194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:06.838194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:06.838194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:06.838194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:06.839071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:06.839071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:50:06.839071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:06.839071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:06.839071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:06.839071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:06.839071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:06.839071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:06.839071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:06.839071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:06.839071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:50:06.839071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:50:06.839071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 22:50:06.839071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:06.841837 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 506/1002] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:50:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:50:06.844335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff4268-79f0-4478-84ba-c195fe4efd9f None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:06.872447 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bec125-11f0-4315-bf41-e908535ca671 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:06.905364 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432c880-61a9-4b00-a2e4-ed3d692e5028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:06.906024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432c880-61a9-4b00-a2e4-ed3d692e5028 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:06.906329 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432c880-61a9-4b00-a2e4-ed3d692e5028 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:06.906602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432c880-61a9-4b00-a2e4-ed3d692e5028 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:06.907295 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432c880-61a9-4b00-a2e4-ed3d692e5028 None admin] RBAC: Adding query filter params (name=image) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:06.907598 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432c880-61a9-4b00-a2e4-ed3d692e5028 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:06.908945 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432c880-61a9-4b00-a2e4-ed3d692e5028 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:06.914234 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 497/1003] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:50:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:06.919457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432c880-61a9-4b00-a2e4-ed3d692e5028 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:06.951976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff4268-79f0-4478-84ba-c195fe4efd9f None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:06.984798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7aaa2-6bb0-4aea-aa8a-d5a1229d6219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:06.985457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7aaa2-6bb0-4aea-aa8a-d5a1229d6219 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:06.985738 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7aaa2-6bb0-4aea-aa8a-d5a1229d6219 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:06.986013 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7aaa2-6bb0-4aea-aa8a-d5a1229d6219 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:06.986759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d7aaa2-6bb0-4aea-aa8a-d5a1229d6219 None admin] RBAC: Adding query filter params (name=compute) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:06.987062 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d7aaa2-6bb0-4aea-aa8a-d5a1229d6219 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:06.989048 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d7aaa2-6bb0-4aea-aa8a-d5a1229d6219 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:06.995955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 507/1004] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 9 22:50:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:07.001218 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7aaa2-6bb0-4aea-aa8a-d5a1229d6219 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:07.021059 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432c880-61a9-4b00-a2e4-ed3d692e5028 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:07.061684 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8acfb98-1a19-4347-9cbc-993fb40f5fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:07.062403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acfb98-1a19-4347-9cbc-993fb40f5fd4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:07.062681 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acfb98-1a19-4347-9cbc-993fb40f5fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:07.062957 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acfb98-1a19-4347-9cbc-993fb40f5fd4 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:07.063668 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8acfb98-1a19-4347-9cbc-993fb40f5fd4 None admin] RBAC: Adding query filter params (type=image) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:07.063971 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8acfb98-1a19-4347-9cbc-993fb40f5fd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:07.065359 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8acfb98-1a19-4347-9cbc-993fb40f5fd4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:07.070299 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 498/1005] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:50:06 2022] GET /identity/v3/services?type=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:07.115696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7aaa2-6bb0-4aea-aa8a-d5a1229d6219 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:07.150221 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a340d4-83ba-4d48-ad2c-bea5e8c5d935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:07.150899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a340d4-83ba-4d48-ad2c-bea5e8c5d935 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:07.151175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a340d4-83ba-4d48-ad2c-bea5e8c5d935 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:07.151448 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a340d4-83ba-4d48-ad2c-bea5e8c5d935 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:07.152166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a340d4-83ba-4d48-ad2c-bea5e8c5d935 None admin] RBAC: Adding query filter params (type=compute) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:07.152472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a340d4-83ba-4d48-ad2c-bea5e8c5d935 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:07.153866 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a340d4-83ba-4d48-ad2c-bea5e8c5d935 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:07.163302 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 508/1006] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 9 22:50:07 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:07.168550 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8acfb98-1a19-4347-9cbc-993fb40f5fd4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:07.245790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8517b1-4457-43a2-823e-5ccffc35ddbb None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:07.246167 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8517b1-4457-43a2-823e-5ccffc35ddbb None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:07.246396 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8517b1-4457-43a2-823e-5ccffc35ddbb None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:07.247768 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 509/1007] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:50:07.253017 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b335153-4a6b-45d3-a2ca-5bf235c93c02 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:07.253301 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b335153-4a6b-45d3-a2ca-5bf235c93c02 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:07.253578 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b335153-4a6b-45d3-a2ca-5bf235c93c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:07.272119 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-9b335153-4a6b-45d3-a2ca-5bf235c93c02 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:07.273232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9b335153-4a6b-45d3-a2ca-5bf235c93c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:07.274261 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8acfb98-1a19-4347-9cbc-993fb40f5fd4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:07.309106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0722f0-da82-4e73-9adf-fc80dfb5f2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:07.309804 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0722f0-da82-4e73-9adf-fc80dfb5f2dc None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:07.310086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0722f0-da82-4e73-9adf-fc80dfb5f2dc None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:07.310432 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0722f0-da82-4e73-9adf-fc80dfb5f2dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:07.311174 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0722f0-da82-4e73-9adf-fc80dfb5f2dc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:07.312532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0722f0-da82-4e73-9adf-fc80dfb5f2dc None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:07.342332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 499/1008] 10.222.0.23 () {60 vars in 1144 bytes} [Mon May 9 22:50:07 2022] POST /identity/v3/endpoints => generated 334 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:50:07.347184 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 510/1009] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:07 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 22:50:07.370730 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0722f0-da82-4e73-9adf-fc80dfb5f2dc None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:07.474329 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0722f0-da82-4e73-9adf-fc80dfb5f2dc None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:07.508362 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d36bf-5433-4ed8-b6bf-9abb74c06ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:07.509169 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d36bf-5433-4ed8-b6bf-9abb74c06ce0 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:07.509447 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d36bf-5433-4ed8-b6bf-9abb74c06ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:07.510022 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d36bf-5433-4ed8-b6bf-9abb74c06ce0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:07.510384 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d36bf-5433-4ed8-b6bf-9abb74c06ce0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:07.511755 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d36bf-5433-4ed8-b6bf-9abb74c06ce0 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:07.524428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-3a3d36bf-5433-4ed8-b6bf-9abb74c06ce0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 22:50:07.524428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:07.524428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:50:07.524428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:50:07.524428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:50:07.524428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:50:07.524428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:50:07.524428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:50:07.524428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 22:50:07.524428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:07.524428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:50:07.524428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:07.524428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:07.524428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:07.524428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:07.524428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:07.524428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:07.525433 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:07.525433 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:07.525433 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:07.525433 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:07.525433 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:07.525433 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:07.525433 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:50:07.525433 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:50:07.525433 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:50:07.525433 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:50:07.525433 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:07.525433 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:07.525433 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:07.525433 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:07.525433 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:07.525433 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:07.526323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:07.526323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:50:07.526323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:07.526323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:07.526323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:07.526323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:07.526323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:07.526323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:07.526323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:07.526323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:07.526323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:50:07.526323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:50:07.526323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 22:50:07.526323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:07.526323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 500/1010] 10.222.0.23 () {56 vars in 1126 bytes} [Mon May 9 22:50:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:50:07.533814 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b335153-4a6b-45d3-a2ca-5bf235c93c02 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:07.633546 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b335153-4a6b-45d3-a2ca-5bf235c93c02 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:07.671168 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf85e08-ab9d-4e0d-8d91-dc377a5ec9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:07.671825 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf85e08-ab9d-4e0d-8d91-dc377a5ec9f1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:07.672101 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf85e08-ab9d-4e0d-8d91-dc377a5ec9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:07.672371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf85e08-ab9d-4e0d-8d91-dc377a5ec9f1 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:07.673057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf85e08-ab9d-4e0d-8d91-dc377a5ec9f1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:07.673355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf85e08-ab9d-4e0d-8d91-dc377a5ec9f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:07.674742 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf85e08-ab9d-4e0d-8d91-dc377a5ec9f1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:07.679545 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 511/1011] 10.222.0.23 () {56 vars in 1122 bytes} [Mon May 9 22:50:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:07.685446 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d36bf-5433-4ed8-b6bf-9abb74c06ce0 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:07.779997 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d36bf-5433-4ed8-b6bf-9abb74c06ce0 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:07.811929 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4529e515-3ba3-41ff-bb70-1f33f950c9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:07.812582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529e515-3ba3-41ff-bb70-1f33f950c9e1 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:07.812872 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529e515-3ba3-41ff-bb70-1f33f950c9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:07.813142 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529e515-3ba3-41ff-bb70-1f33f950c9e1 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:07.813786 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4529e515-3ba3-41ff-bb70-1f33f950c9e1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:07.814077 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4529e515-3ba3-41ff-bb70-1f33f950c9e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:07.815520 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4529e515-3ba3-41ff-bb70-1f33f950c9e1 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:07.821310 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 501/1012] 10.222.0.23 () {56 vars in 1122 bytes} [Mon May 9 22:50:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:07.826838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf85e08-ab9d-4e0d-8d91-dc377a5ec9f1 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:07.920501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf85e08-ab9d-4e0d-8d91-dc377a5ec9f1 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:07.953145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c871bc-4202-4934-b1e6-8fc573ae9792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:07.953795 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c871bc-4202-4934-b1e6-8fc573ae9792 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:07.954068 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c871bc-4202-4934-b1e6-8fc573ae9792 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:07.954364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c871bc-4202-4934-b1e6-8fc573ae9792 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:07.956346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c871bc-4202-4934-b1e6-8fc573ae9792 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:07.956468 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c871bc-4202-4934-b1e6-8fc573ae9792 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:07.972480 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 512/1013] 10.222.0.23 () {60 vars in 1144 bytes} [Mon May 9 22:50:07 2022] POST /identity/v3/endpoints => generated 325 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:50:09.310231 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679196f-3b10-4858-8236-610760e52a4b None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:09.310763 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679196f-3b10-4858-8236-610760e52a4b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:09.310763 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679196f-3b10-4858-8236-610760e52a4b None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:09.312122 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 502/1014] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:50:09.318064 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f6ef7-3a42-4d4d-8a98-dde09e9f8ee3 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:09.318535 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f6ef7-3a42-4d4d-8a98-dde09e9f8ee3 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:09.318697 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f6ef7-3a42-4d4d-8a98-dde09e9f8ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:09.335253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f81f6ef7-3a42-4d4d-8a98-dde09e9f8ee3 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:09.336414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f81f6ef7-3a42-4d4d-8a98-dde09e9f8ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:09.404326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 513/1015] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:09 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:50:09.427060 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9679196f-3b10-4858-8236-610760e52a4b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:09.525127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9679196f-3b10-4858-8236-610760e52a4b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:09.558564 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5f6e1-255f-4fcf-986e-e22372e89014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:09.559215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5f6e1-255f-4fcf-986e-e22372e89014 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:09.559496 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5f6e1-255f-4fcf-986e-e22372e89014 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:09.559765 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5f6e1-255f-4fcf-986e-e22372e89014 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:09.560554 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5f6e1-255f-4fcf-986e-e22372e89014 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:09.561846 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5f6e1-255f-4fcf-986e-e22372e89014 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:09.578194 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 503/1016] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 9 22:50:09 2022] POST /identity/v3/services => generated 242 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:50:09.698821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dab108-2fd7-4068-af7e-62103df5f60c None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:09.699146 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dab108-2fd7-4068-af7e-62103df5f60c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:09.699373 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dab108-2fd7-4068-af7e-62103df5f60c None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:09.700786 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 514/1017] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:50:09.707166 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99697b-c6d4-4689-ac12-bf9613a89029 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:09.707488 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99697b-c6d4-4689-ac12-bf9613a89029 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:09.707731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99697b-c6d4-4689-ac12-bf9613a89029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:09.725865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ee99697b-c6d4-4689-ac12-bf9613a89029 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:09.727143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ee99697b-c6d4-4689-ac12-bf9613a89029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:09.802740 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 504/1018] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:50:09.826383 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dab108-2fd7-4068-af7e-62103df5f60c None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:09.930952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dab108-2fd7-4068-af7e-62103df5f60c None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:09.964799 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100cf42-7935-4034-a32e-37a4c8d4f620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:09.965602 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100cf42-7935-4034-a32e-37a4c8d4f620 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:09.965880 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100cf42-7935-4034-a32e-37a4c8d4f620 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:09.966202 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100cf42-7935-4034-a32e-37a4c8d4f620 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:09.975949 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100cf42-7935-4034-a32e-37a4c8d4f620 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:50:09.984308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-9100cf42-7935-4034-a32e-37a4c8d4f620 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 22:50:09.984308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:09.984308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:09.984308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:09.984308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:09.984308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:09.984308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:09.984308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:09.984308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:09.984308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:09.984308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:09.984308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:09.984308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:50:09.984308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:50:09.984308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:50:09.984308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:50:09.985250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:50:09.985250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:50:09.985250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:50:09.985250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:50:09.985250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:09.985250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:09.985250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:09.985250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:09.985250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:09.985250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:09.985250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:09.985250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:50:09.985250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:09.985250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:09.985250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:09.985250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:09.986103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:09.986103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:09.986103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:09.986103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:09.986103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:50:09.986103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:50:09.986103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:50:09.986103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:50:09.986103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:50:09.986103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:50:09.986103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 22:50:09.986103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:09.987378 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 515/1019] 10.222.0.23 () {56 vars in 1120 bytes} [Mon May 9 22:50:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:50:09.991661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99697b-c6d4-4689-ac12-bf9613a89029 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:10.091387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99697b-c6d4-4689-ac12-bf9613a89029 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:10.125670 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131b3fa-d249-4ab7-a998-a902490cc74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:10.126429 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131b3fa-d249-4ab7-a998-a902490cc74a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:10.126711 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131b3fa-d249-4ab7-a998-a902490cc74a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:10.126983 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131b3fa-d249-4ab7-a998-a902490cc74a None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:10.127718 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131b3fa-d249-4ab7-a998-a902490cc74a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:10.128020 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131b3fa-d249-4ab7-a998-a902490cc74a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:10.129405 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131b3fa-d249-4ab7-a998-a902490cc74a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:10.141562 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 505/1020] 10.222.0.23 () {56 vars in 1115 bytes} [Mon May 9 22:50:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:10.143276 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100cf42-7935-4034-a32e-37a4c8d4f620 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:10.202728 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88bbe6-680f-4de9-92ce-28fa983699fb None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:10.203303 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88bbe6-680f-4de9-92ce-28fa983699fb None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:10.203795 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88bbe6-680f-4de9-92ce-28fa983699fb None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:10.205725 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 506/1021] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:50:10.210935 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df90c59-108c-4774-bdb9-da3d309b0719 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:10.211478 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df90c59-108c-4774-bdb9-da3d309b0719 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:10.211936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df90c59-108c-4774-bdb9-da3d309b0719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:10.231344 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-1df90c59-108c-4774-bdb9-da3d309b0719 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:10.232540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1df90c59-108c-4774-bdb9-da3d309b0719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:10.241249 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100cf42-7935-4034-a32e-37a4c8d4f620 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:10.280878 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad15ba-077b-4e59-a777-c44fedb3e543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:10.281663 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad15ba-077b-4e59-a777-c44fedb3e543 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:10.281941 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad15ba-077b-4e59-a777-c44fedb3e543 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:10.282241 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad15ba-077b-4e59-a777-c44fedb3e543 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:10.293534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad15ba-077b-4e59-a777-c44fedb3e543 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:10.294996 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad15ba-077b-4e59-a777-c44fedb3e543 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:10.305889 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 507/1022] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:50:10.315353 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-95ad15ba-077b-4e59-a777-c44fedb3e543 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:50:10.315353 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:10.315353 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:10.315353 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:10.315353 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:10.315353 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:10.315353 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:10.315353 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:10.315353 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:10.315353 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:10.315353 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:10.315353 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:10.315353 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:50:10.315353 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:50:10.315353 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:50:10.316303 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:50:10.316303 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:10.316303 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:10.316303 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:10.316303 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:10.316303 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:10.316303 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:10.316303 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:10.316303 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:50:10.316303 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:10.316303 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:10.316303 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:10.316303 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:10.316303 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:10.316303 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:10.316303 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:10.316303 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:10.317234 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:50:10.317234 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:50:10.317234 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:50:10.317234 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:10.318029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 516/1023] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:50:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:50:10.322061 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df90c59-108c-4774-bdb9-da3d309b0719 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:10.328753 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad15ba-077b-4e59-a777-c44fedb3e543 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:10.417917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df90c59-108c-4774-bdb9-da3d309b0719 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:10.427647 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad15ba-077b-4e59-a777-c44fedb3e543 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:10.452282 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd4172-de4e-4f4c-af2f-16320d77f79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:10.452988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd4172-de4e-4f4c-af2f-16320d77f79c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:10.453275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd4172-de4e-4f4c-af2f-16320d77f79c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:10.453550 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd4172-de4e-4f4c-af2f-16320d77f79c None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:10.454292 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd4172-de4e-4f4c-af2f-16320d77f79c None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:10.454596 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd4172-de4e-4f4c-af2f-16320d77f79c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:10.456383 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd4172-de4e-4f4c-af2f-16320d77f79c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:10.460894 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32c7d9-3144-43ff-9170-77d42084fa0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:10.461678 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32c7d9-3144-43ff-9170-77d42084fa0e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:10.461958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32c7d9-3144-43ff-9170-77d42084fa0e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:10.462262 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32c7d9-3144-43ff-9170-77d42084fa0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:10.463426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32c7d9-3144-43ff-9170-77d42084fa0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:10.464842 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32c7d9-3144-43ff-9170-77d42084fa0e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:10.466645 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 517/1024] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:50:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:10.471620 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 508/1025] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:10.471990 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32c7d9-3144-43ff-9170-77d42084fa0e None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:10.472558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd4172-de4e-4f4c-af2f-16320d77f79c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:10.573499 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32c7d9-3144-43ff-9170-77d42084fa0e None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:10.573848 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd4172-de4e-4f4c-af2f-16320d77f79c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:10.609835 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0fe34-5db1-49b0-8130-5dcb5333ce5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:10.610390 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec637b-ed67-416c-a451-9085e133a123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:10.610512 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0fe34-5db1-49b0-8130-5dcb5333ce5a None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:10.610791 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0fe34-5db1-49b0-8130-5dcb5333ce5a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:10.611061 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0fe34-5db1-49b0-8130-5dcb5333ce5a None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:10.611161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec637b-ed67-416c-a451-9085e133a123 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:10.611431 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec637b-ed67-416c-a451-9085e133a123 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:10.611702 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec637b-ed67-416c-a451-9085e133a123 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:10.611797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b0fe34-5db1-49b0-8130-5dcb5333ce5a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:10.613834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b0fe34-5db1-49b0-8130-5dcb5333ce5a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:10.619999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec637b-ed67-416c-a451-9085e133a123 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:10.624408 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec637b-ed67-416c-a451-9085e133a123 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:10.639685 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-46ec637b-ed67-416c-a451-9085e133a123 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 22:50:10.639685 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:10.639685 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:10.639685 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:10.639685 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:10.639685 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:10.639685 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:10.639685 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:10.639685 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:10.639685 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:10.639685 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:10.639685 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:10.639685 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:50:10.639685 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:50:10.639685 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:50:10.639685 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:50:10.640863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:10.640863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:10.640863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:50:10.640863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:50:10.640863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:50:10.640863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:50:10.640863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:10.640863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:10.640863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:10.640863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:10.640863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:10.640863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:50:10.640863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:10.640863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:10.640863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:10.640863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:10.640863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:10.640863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:10.641958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:10.641958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:10.641958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:50:10.641958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:50:10.641958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:50:10.641958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:50:10.641958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:50:10.641958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:50:10.641958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 22:50:10.641958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:10.641958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 509/1026] 10.222.0.23 () {56 vars in 1093 bytes} [Mon May 9 22:50:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:50:10.646379 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec637b-ed67-416c-a451-9085e133a123 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:10.658374 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 518/1027] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 9 22:50:10 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 22:50:10.755195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec637b-ed67-416c-a451-9085e133a123 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:10.790400 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc5352-7adc-4227-af78-55ba2196b9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:10.791084 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc5352-7adc-4227-af78-55ba2196b9e9 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:10.791365 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc5352-7adc-4227-af78-55ba2196b9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:10.791641 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc5352-7adc-4227-af78-55ba2196b9e9 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:10.792482 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc5352-7adc-4227-af78-55ba2196b9e9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:10.792790 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc5352-7adc-4227-af78-55ba2196b9e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:10.794257 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc5352-7adc-4227-af78-55ba2196b9e9 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:10.815490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 510/1028] 10.222.0.23 () {56 vars in 1133 bytes} [Mon May 9 22:50:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:10.820680 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0fe34-5db1-49b0-8130-5dcb5333ce5a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:10.922630 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0fe34-5db1-49b0-8130-5dcb5333ce5a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:10.954605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695c1cd-3a94-4b1a-bf19-e28057024c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:10.955251 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695c1cd-3a94-4b1a-bf19-e28057024c3b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:10.955525 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695c1cd-3a94-4b1a-bf19-e28057024c3b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:10.955792 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695c1cd-3a94-4b1a-bf19-e28057024c3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:11.018605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695c1cd-3a94-4b1a-bf19-e28057024c3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:11.020483 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695c1cd-3a94-4b1a-bf19-e28057024c3b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:11.116907 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 519/1029] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:50:10 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 22:50:11.118471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc5352-7adc-4227-af78-55ba2196b9e9 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:11.212332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc5352-7adc-4227-af78-55ba2196b9e9 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:11.245459 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9592e72-51ca-445c-a879-aac2c5ddf6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:11.246267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9592e72-51ca-445c-a879-aac2c5ddf6cb None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:11.246540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9592e72-51ca-445c-a879-aac2c5ddf6cb None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:11.246804 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9592e72-51ca-445c-a879-aac2c5ddf6cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:11.256402 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9592e72-51ca-445c-a879-aac2c5ddf6cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:11.257823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9592e72-51ca-445c-a879-aac2c5ddf6cb None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:11.275032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-e9592e72-51ca-445c-a879-aac2c5ddf6cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:50:11.275032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:11.275032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:11.275032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:11.275032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:11.275032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:11.275032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:11.275032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:11.275032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:11.275032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:11.275032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:11.275032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:11.275032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:50:11.275032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:50:11.275032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:50:11.275946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:50:11.275946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:11.275946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:11.275946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:11.275946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:11.275946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:11.275946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:11.275946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:11.275946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:50:11.275946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:11.275946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:11.275946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:11.275946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:11.275946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:11.275946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:11.275946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:11.275946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:11.276866 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:50:11.276866 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:50:11.276866 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:50:11.276866 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:11.276866 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 511/1030] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:50:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:50:11.283085 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695c1cd-3a94-4b1a-bf19-e28057024c3b None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:11.381331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695c1cd-3a94-4b1a-bf19-e28057024c3b None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:11.413363 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a97d8b-3d42-4402-bf65-022fae77ff3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:11.414017 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a97d8b-3d42-4402-bf65-022fae77ff3a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:11.414340 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a97d8b-3d42-4402-bf65-022fae77ff3a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:11.414590 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a97d8b-3d42-4402-bf65-022fae77ff3a None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:11.415290 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a97d8b-3d42-4402-bf65-022fae77ff3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:11.415589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a97d8b-3d42-4402-bf65-022fae77ff3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:11.416933 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a97d8b-3d42-4402-bf65-022fae77ff3a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:11.432252 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 520/1031] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:11.433774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9592e72-51ca-445c-a879-aac2c5ddf6cb None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:11.528093 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9592e72-51ca-445c-a879-aac2c5ddf6cb None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:11.559625 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2083a1-b047-4657-a42b-889423577226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:11.560407 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2083a1-b047-4657-a42b-889423577226 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:11.560684 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2083a1-b047-4657-a42b-889423577226 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:11.560950 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2083a1-b047-4657-a42b-889423577226 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:11.567907 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2083a1-b047-4657-a42b-889423577226 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:11.569319 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2083a1-b047-4657-a42b-889423577226 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:11.582929 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-ce2083a1-b047-4657-a42b-889423577226 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:50:11.582929 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:11.582929 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:11.582929 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:11.582929 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:11.582929 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:11.582929 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:11.582929 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:11.582929 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:11.582929 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:11.582929 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:11.582929 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:11.582929 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:50:11.582929 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:50:11.582929 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:50:11.583817 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:50:11.583817 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:11.583817 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:11.583817 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:11.583817 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:11.583817 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:11.583817 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:11.583817 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:11.583817 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:50:11.583817 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:11.583817 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:11.583817 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:11.583817 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:11.583817 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:11.583817 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:11.583817 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:11.583817 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:11.584907 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:50:11.584907 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:50:11.584907 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:50:11.584907 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:50:11.584907 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:50:11.584907 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:50:11.584907 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:50:11.584907 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:11.584907 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 512/1032] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:50:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:50:11.589783 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a97d8b-3d42-4402-bf65-022fae77ff3a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:11.689191 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a97d8b-3d42-4402-bf65-022fae77ff3a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:11.721227 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc36d6d-cfd0-4873-926d-2296f3753e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:11.721882 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc36d6d-cfd0-4873-926d-2296f3753e65 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:11.722226 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc36d6d-cfd0-4873-926d-2296f3753e65 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:11.722506 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc36d6d-cfd0-4873-926d-2296f3753e65 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:11.723287 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc36d6d-cfd0-4873-926d-2296f3753e65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:11.723601 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc36d6d-cfd0-4873-926d-2296f3753e65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:11.725033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc36d6d-cfd0-4873-926d-2296f3753e65 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:11.742557 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 521/1033] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:50:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:11.744083 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2083a1-b047-4657-a42b-889423577226 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:11.833506 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fafc85-4c58-4ce1-a684-3ead59ce2f34 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:11.833711 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fafc85-4c58-4ce1-a684-3ead59ce2f34 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:11.833961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fafc85-4c58-4ce1-a684-3ead59ce2f34 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:11.835496 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 522/1034] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:50:11.840055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09d408-d9d6-4762-8eed-ea8ceb0658ba None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:11.840341 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09d408-d9d6-4762-8eed-ea8ceb0658ba None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:11.840618 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09d408-d9d6-4762-8eed-ea8ceb0658ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:11.841199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2083a1-b047-4657-a42b-889423577226 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:11.856329 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5d09d408-d9d6-4762-8eed-ea8ceb0658ba None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:11.857496 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5d09d408-d9d6-4762-8eed-ea8ceb0658ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:11.873950 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b15133b-d6e0-442a-854f-39ce6498b97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:11.874637 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15133b-d6e0-442a-854f-39ce6498b97e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:11.874911 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15133b-d6e0-442a-854f-39ce6498b97e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:11.875179 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15133b-d6e0-442a-854f-39ce6498b97e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:11.876066 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b15133b-d6e0-442a-854f-39ce6498b97e None admin] RBAC: Adding query filter params (role.id=8b7c99010273443e88d3e81ce6924349, scope.project.id=0500e378ea374ca692e230bbd8781318, user.id=02fcd7cc86134ce488c7dd75dfb174b8) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:11.876531 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b15133b-d6e0-442a-854f-39ce6498b97e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:11.877686 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b15133b-d6e0-442a-854f-39ce6498b97e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:11.883891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 513/1035] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:50:11 2022] GET /identity/v3/role_assignments?user.id=02fcd7cc86134ce488c7dd75dfb174b8&scope.project.id=0500e378ea374ca692e230bbd8781318&role.id=8b7c99010273443e88d3e81ce6924349 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:11.929094 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 523/1036] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:11 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 22:50:11.951832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b15133b-d6e0-442a-854f-39ce6498b97e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:12.054324 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b15133b-d6e0-442a-854f-39ce6498b97e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:12.088537 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273739b-96ba-48cb-ab2c-dd4fe44fed1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:12.089262 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273739b-96ba-48cb-ab2c-dd4fe44fed1b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:12.089542 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273739b-96ba-48cb-ab2c-dd4fe44fed1b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:12.089816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273739b-96ba-48cb-ab2c-dd4fe44fed1b None admin] PATH_INFO: `/v3/services/image` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:12.090496 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273739b-96ba-48cb-ab2c-dd4fe44fed1b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:12.091849 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273739b-96ba-48cb-ab2c-dd4fe44fed1b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:12.103743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-c273739b-96ba-48cb-ab2c-dd4fe44fed1b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 09 22:50:12.103743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:12.103743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:50:12.103743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:50:12.103743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:50:12.103743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:50:12.103743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:50:12.103743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:50:12.103743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 22:50:12.103743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:12.103743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:50:12.103743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:12.103743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:12.103743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:12.103743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:12.103743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:12.103743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:12.104751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:12.104751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:12.104751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:12.104751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:12.104751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:12.104751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:12.104751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:50:12.104751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:50:12.104751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:50:12.104751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:50:12.104751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:12.104751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:12.104751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:12.104751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:12.104751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:12.104751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:12.105599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:12.105599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:50:12.105599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:12.105599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:12.105599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:12.105599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:12.105599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:12.105599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:12.105599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:12.105599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:12.105599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:50:12.105599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:50:12.105599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 22:50:12.105599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:12.105599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 514/1037] 10.222.0.23 () {56 vars in 1105 bytes} [Mon May 9 22:50:11 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:50:12.113988 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09d408-d9d6-4762-8eed-ea8ceb0658ba None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:12.206056 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09d408-d9d6-4762-8eed-ea8ceb0658ba None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:12.239473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a80e21-8e4e-470b-93c7-c83d816d12b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:12.240131 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a80e21-8e4e-470b-93c7-c83d816d12b9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:12.240410 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a80e21-8e4e-470b-93c7-c83d816d12b9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:12.240680 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a80e21-8e4e-470b-93c7-c83d816d12b9 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:12.241377 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a80e21-8e4e-470b-93c7-c83d816d12b9 None admin] RBAC: Adding query filter params (name=image) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:12.241678 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a80e21-8e4e-470b-93c7-c83d816d12b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:12.243074 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a80e21-8e4e-470b-93c7-c83d816d12b9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:12.252450 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 524/1038] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:50:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:12.253682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273739b-96ba-48cb-ab2c-dd4fe44fed1b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:12.349281 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273739b-96ba-48cb-ab2c-dd4fe44fed1b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:12.382307 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77731e8-cfb6-4c16-aa8d-b2ae2c03aa5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:12.382964 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77731e8-cfb6-4c16-aa8d-b2ae2c03aa5d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:12.383236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77731e8-cfb6-4c16-aa8d-b2ae2c03aa5d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:12.383497 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77731e8-cfb6-4c16-aa8d-b2ae2c03aa5d None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:12.384184 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77731e8-cfb6-4c16-aa8d-b2ae2c03aa5d None admin] RBAC: Adding query filter params (type=image) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:12.384483 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77731e8-cfb6-4c16-aa8d-b2ae2c03aa5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:12.385822 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77731e8-cfb6-4c16-aa8d-b2ae2c03aa5d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:12.395297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 515/1039] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:50:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:12.396613 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a80e21-8e4e-470b-93c7-c83d816d12b9 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:12.493454 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a80e21-8e4e-470b-93c7-c83d816d12b9 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:12.526663 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff1eaa-078c-4906-a169-ae3a29b19ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:12.527318 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff1eaa-078c-4906-a169-ae3a29b19ef3 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:12.527594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff1eaa-078c-4906-a169-ae3a29b19ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:12.527863 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff1eaa-078c-4906-a169-ae3a29b19ef3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:12.528643 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff1eaa-078c-4906-a169-ae3a29b19ef3 None admin] RBAC: Adding query filter params (interface=public, service_id=1b825e0a0e2f45cbab4c125322f81e3e, region_id=RegionOne) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:12.528949 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff1eaa-078c-4906-a169-ae3a29b19ef3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:12.530331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff1eaa-078c-4906-a169-ae3a29b19ef3 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:12.538377 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 525/1040] 10.222.0.23 () {56 vars in 1251 bytes} [Mon May 9 22:50:12 2022] GET /identity/v3/endpoints?service_id=1b825e0a0e2f45cbab4c125322f81e3e&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:12.541518 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77731e8-cfb6-4c16-aa8d-b2ae2c03aa5d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:12.636345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77731e8-cfb6-4c16-aa8d-b2ae2c03aa5d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:12.668201 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1f0d3-eb2e-4eaa-915d-971d10af9523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:12.668851 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1f0d3-eb2e-4eaa-915d-971d10af9523 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:12.669126 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1f0d3-eb2e-4eaa-915d-971d10af9523 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:12.669393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1f0d3-eb2e-4eaa-915d-971d10af9523 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:12.670046 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1f0d3-eb2e-4eaa-915d-971d10af9523 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:12.671430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1f0d3-eb2e-4eaa-915d-971d10af9523 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:12.677870 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 516/1041] 10.222.0.23 () {56 vars in 1087 bytes} [Mon May 9 22:50:12 2022] GET /identity/v3/services => generated 1957 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 22:50:12.949281 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cde06f-a6d5-44de-8022-1903e1943b47 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:12.949623 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cde06f-a6d5-44de-8022-1903e1943b47 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:12.949797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cde06f-a6d5-44de-8022-1903e1943b47 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:12.951304 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 526/1042] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:50:12.956300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5f739-4f71-4f8e-a3cf-9b262ce07445 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:12.956616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5f739-4f71-4f8e-a3cf-9b262ce07445 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:12.956892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5f739-4f71-4f8e-a3cf-9b262ce07445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:12.974267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-aeb5f739-4f71-4f8e-a3cf-9b262ce07445 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:12.975437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-aeb5f739-4f71-4f8e-a3cf-9b262ce07445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:13.042811 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 517/1043] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:50:13.067116 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cde06f-a6d5-44de-8022-1903e1943b47 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:13.163802 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cde06f-a6d5-44de-8022-1903e1943b47 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:13.197447 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199c4a9-ab58-47e2-8959-c8cc45b9460c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:13.198262 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199c4a9-ab58-47e2-8959-c8cc45b9460c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:13.198541 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199c4a9-ab58-47e2-8959-c8cc45b9460c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:13.198811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199c4a9-ab58-47e2-8959-c8cc45b9460c None admin] PATH_INFO: `/v3/domains/default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:13.200028 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199c4a9-ab58-47e2-8959-c8cc45b9460c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:13.201411 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199c4a9-ab58-47e2-8959-c8cc45b9460c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:13.203246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 527/1044] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:50:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:13.209212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5f739-4f71-4f8e-a3cf-9b262ce07445 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:13.303146 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5f739-4f71-4f8e-a3cf-9b262ce07445 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:13.334822 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1521e13-53d7-4ebf-b50c-0c63d6885553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:13.335487 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1521e13-53d7-4ebf-b50c-0c63d6885553 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:13.335764 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1521e13-53d7-4ebf-b50c-0c63d6885553 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:13.336036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1521e13-53d7-4ebf-b50c-0c63d6885553 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:13.336738 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1521e13-53d7-4ebf-b50c-0c63d6885553 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:13.338206 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1521e13-53d7-4ebf-b50c-0c63d6885553 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:13.365983 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 518/1045] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 9 22:50:13 2022] POST /identity/v3/users => generated 285 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 22:50:14.094740 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5591017-6022-413f-8557-eb8d5655173b None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:14.095270 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5591017-6022-413f-8557-eb8d5655173b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:14.095270 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5591017-6022-413f-8557-eb8d5655173b None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:14.096663 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 528/1046] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:50:14.101834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aea99e-1e25-4c4f-b32c-6d06e54aa807 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:14.102261 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aea99e-1e25-4c4f-b32c-6d06e54aa807 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:14.102451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aea99e-1e25-4c4f-b32c-6d06e54aa807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:14.119597 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-77aea99e-1e25-4c4f-b32c-6d06e54aa807 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:14.120761 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-77aea99e-1e25-4c4f-b32c-6d06e54aa807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:14.184959 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 519/1047] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:50:14.207772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5591017-6022-413f-8557-eb8d5655173b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:14.305247 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5591017-6022-413f-8557-eb8d5655173b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:14.338380 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a10b4-bae6-409f-a0a8-804cd1b45d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:14.339180 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a10b4-bae6-409f-a0a8-804cd1b45d33 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:14.339457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a10b4-bae6-409f-a0a8-804cd1b45d33 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:14.339726 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a10b4-bae6-409f-a0a8-804cd1b45d33 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:14.349443 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a10b4-bae6-409f-a0a8-804cd1b45d33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:50:14.357561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-3d7a10b4-bae6-409f-a0a8-804cd1b45d33 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 22:50:14.357561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:14.357561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:14.357561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:14.357561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:14.357561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:14.357561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:14.357561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:14.357561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:14.357561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:14.357561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:14.357561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:14.357561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:50:14.357561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:50:14.357561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:50:14.357561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:50:14.358703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:50:14.358703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:50:14.358703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:50:14.358703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:50:14.358703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:14.358703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:14.358703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:14.358703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:14.358703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:14.358703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:14.358703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:14.358703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:50:14.358703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:14.358703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:14.358703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:14.358703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:14.359853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:14.359853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:14.359853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:14.359853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:14.359853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:50:14.359853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:50:14.359853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:50:14.359853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:50:14.359853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:50:14.359853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:50:14.359853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 22:50:14.359853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:14.362431 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 529/1048] 10.222.0.23 () {56 vars in 1120 bytes} [Mon May 9 22:50:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:50:14.364861 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aea99e-1e25-4c4f-b32c-6d06e54aa807 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:14.460437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aea99e-1e25-4c4f-b32c-6d06e54aa807 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:14.492449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4934c-021f-4b76-8fd5-059a9fb25229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:14.493111 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4934c-021f-4b76-8fd5-059a9fb25229 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:14.493386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4934c-021f-4b76-8fd5-059a9fb25229 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:14.493656 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4934c-021f-4b76-8fd5-059a9fb25229 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:14.494421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4934c-021f-4b76-8fd5-059a9fb25229 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:14.494724 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4934c-021f-4b76-8fd5-059a9fb25229 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:14.496075 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4934c-021f-4b76-8fd5-059a9fb25229 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:14.509001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 520/1049] 10.222.0.23 () {56 vars in 1115 bytes} [Mon May 9 22:50:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:14.510330 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a10b4-bae6-409f-a0a8-804cd1b45d33 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:14.607432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a10b4-bae6-409f-a0a8-804cd1b45d33 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:14.640982 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762270f-1f9a-4c09-b97e-90f6a8bd4067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:14.641782 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762270f-1f9a-4c09-b97e-90f6a8bd4067 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:14.642060 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762270f-1f9a-4c09-b97e-90f6a8bd4067 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:14.642364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762270f-1f9a-4c09-b97e-90f6a8bd4067 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:14.653346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762270f-1f9a-4c09-b97e-90f6a8bd4067 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:14.654832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762270f-1f9a-4c09-b97e-90f6a8bd4067 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:14.671293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-2762270f-1f9a-4c09-b97e-90f6a8bd4067 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:50:14.671293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:14.671293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:14.671293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:14.671293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:14.671293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:14.671293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:14.671293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:14.671293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:14.671293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:14.671293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:14.671293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:14.671293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:50:14.671293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:50:14.671293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:50:14.672182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:50:14.672182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:14.672182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:14.672182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:14.672182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:14.672182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:14.672182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:14.672182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:14.672182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:50:14.672182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:14.672182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:14.672182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:14.672182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:14.672182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:14.672182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:14.672182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:14.672182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:14.673100 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:50:14.673100 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:50:14.673100 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:50:14.673100 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:14.673864 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 530/1050] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:50:14.678745 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4934c-021f-4b76-8fd5-059a9fb25229 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:14.780395 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4934c-021f-4b76-8fd5-059a9fb25229 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:14.811814 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64708d-c860-4b03-a8fc-9aa1b325efca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:14.812464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64708d-c860-4b03-a8fc-9aa1b325efca None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:14.812740 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64708d-c860-4b03-a8fc-9aa1b325efca None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:14.813010 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64708d-c860-4b03-a8fc-9aa1b325efca None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:14.813716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a64708d-c860-4b03-a8fc-9aa1b325efca None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:14.814020 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a64708d-c860-4b03-a8fc-9aa1b325efca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:14.815401 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a64708d-c860-4b03-a8fc-9aa1b325efca None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:14.827479 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 521/1051] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:14.832803 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762270f-1f9a-4c09-b97e-90f6a8bd4067 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:14.914904 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55af16a-e9f1-420e-8392-63f1c07390fe None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:14.915092 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55af16a-e9f1-420e-8392-63f1c07390fe None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:14.915374 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55af16a-e9f1-420e-8392-63f1c07390fe None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:14.916863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 522/1052] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:50:14.921613 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf106b7-f2ed-4982-8924-fce560b2c18b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:14.921901 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf106b7-f2ed-4982-8924-fce560b2c18b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:14.922203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf106b7-f2ed-4982-8924-fce560b2c18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:14.928137 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762270f-1f9a-4c09-b97e-90f6a8bd4067 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:14.938928 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-0bf106b7-f2ed-4982-8924-fce560b2c18b None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:14.940107 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0bf106b7-f2ed-4982-8924-fce560b2c18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:14.961276 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5afed78-d358-4821-8973-2f9f6f5ac030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:14.962061 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afed78-d358-4821-8973-2f9f6f5ac030 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:14.962372 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afed78-d358-4821-8973-2f9f6f5ac030 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:14.962645 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afed78-d358-4821-8973-2f9f6f5ac030 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:14.970073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5afed78-d358-4821-8973-2f9f6f5ac030 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:14.971596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5afed78-d358-4821-8973-2f9f6f5ac030 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:14.986014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-a5afed78-d358-4821-8973-2f9f6f5ac030 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 22:50:14.986014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:14.986014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:14.986014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:14.986014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:14.986014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:14.986014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:14.986014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:14.986014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:14.986014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:14.986014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:14.986014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:14.986014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:50:14.986014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:50:14.986014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:50:14.986014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:50:14.987232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:14.987232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:14.987232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:50:14.987232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:50:14.987232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:50:14.987232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:50:14.987232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:14.987232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:14.987232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:14.987232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:14.987232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:14.987232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:50:14.987232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:14.987232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:14.987232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:14.987232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:14.987232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:14.987232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:14.988470 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:14.988470 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:14.988470 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:50:14.988470 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:50:14.988470 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:50:14.988470 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:50:14.988470 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:50:14.988470 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:50:14.988470 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 22:50:14.988470 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:14.991838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 531/1053] 10.222.0.23 () {56 vars in 1093 bytes} [Mon May 9 22:50:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:50:14.992651 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5afed78-d358-4821-8973-2f9f6f5ac030 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:15.011849 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 523/1054] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 22:50:15.035226 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf106b7-f2ed-4982-8924-fce560b2c18b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:15.093441 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5afed78-d358-4821-8973-2f9f6f5ac030 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:15.126493 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b40c4-e5d9-4641-855e-de9c1c869413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:15.127248 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b40c4-e5d9-4641-855e-de9c1c869413 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:15.127462 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b40c4-e5d9-4641-855e-de9c1c869413 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:15.127697 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b40c4-e5d9-4641-855e-de9c1c869413 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:15.129055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b40c4-e5d9-4641-855e-de9c1c869413 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:15.129361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b40c4-e5d9-4641-855e-de9c1c869413 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:15.130806 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b40c4-e5d9-4641-855e-de9c1c869413 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:15.131363 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf106b7-f2ed-4982-8924-fce560b2c18b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:15.148277 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 532/1055] 10.222.0.23 () {56 vars in 1133 bytes} [Mon May 9 22:50:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:15.153659 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b40c4-e5d9-4641-855e-de9c1c869413 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:15.167732 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc91b90-d245-400f-871f-b9b1f3aa338e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:15.168552 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc91b90-d245-400f-871f-b9b1f3aa338e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:15.168902 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc91b90-d245-400f-871f-b9b1f3aa338e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:15.169243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc91b90-d245-400f-871f-b9b1f3aa338e None admin] PATH_INFO: `/v3/services/image` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:15.169961 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc91b90-d245-400f-871f-b9b1f3aa338e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:15.171414 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc91b90-d245-400f-871f-b9b1f3aa338e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:15.183342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-5dc91b90-d245-400f-871f-b9b1f3aa338e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 09 22:50:15.183342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:15.183342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:50:15.183342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:50:15.183342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:50:15.183342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:50:15.183342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:50:15.183342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:50:15.183342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 22:50:15.183342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:15.183342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:50:15.183342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:15.183342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:15.183342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:15.183342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:15.183342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:15.183342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:15.184366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:15.184366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:15.184366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:15.184366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:15.184366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:15.184366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:15.184366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:50:15.184366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:50:15.184366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:50:15.184366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:50:15.184366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:15.184366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:15.184366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:15.184366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:15.184366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:15.184366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:15.185638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:15.185638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:50:15.185638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:15.185638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:15.185638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:15.185638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:15.185638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:15.185638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:15.185638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:15.185638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:15.185638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:50:15.185638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:50:15.185638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 22:50:15.185638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:15.189584 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 524/1056] 10.222.0.23 () {56 vars in 1105 bytes} [Mon May 9 22:50:15 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:50:15.190252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc91b90-d245-400f-871f-b9b1f3aa338e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:15.255016 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b40c4-e5d9-4641-855e-de9c1c869413 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:15.286758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc91b90-d245-400f-871f-b9b1f3aa338e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:15.287768 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033e250-0569-46de-9bc3-40a6ffd8879a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:15.288578 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033e250-0569-46de-9bc3-40a6ffd8879a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:15.288854 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033e250-0569-46de-9bc3-40a6ffd8879a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:15.289153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033e250-0569-46de-9bc3-40a6ffd8879a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:15.298998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033e250-0569-46de-9bc3-40a6ffd8879a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:15.301093 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033e250-0569-46de-9bc3-40a6ffd8879a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:15.317950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-3033e250-0569-46de-9bc3-40a6ffd8879a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:50:15.317950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:15.317950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:15.317950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:15.317950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:15.317950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:15.317950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:15.317950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:15.317950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:15.317950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:15.317950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:15.317950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:15.317950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:50:15.317950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:50:15.317950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:50:15.318887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:50:15.318887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:15.318887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:15.318887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:15.318887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:15.318887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:15.318887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:15.318887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:15.318887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:50:15.318887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:15.318887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:15.318887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:15.318887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:15.318887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:15.318887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:15.318887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:15.318887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:15.319813 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:50:15.319813 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:50:15.319813 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:50:15.319813 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:15.319813 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 533/1057] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:50:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:50:15.323931 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101859b5-af08-4621-91d4-29745fd3c1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:15.324577 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033e250-0569-46de-9bc3-40a6ffd8879a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:15.325132 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101859b5-af08-4621-91d4-29745fd3c1a6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:15.325598 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101859b5-af08-4621-91d4-29745fd3c1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:15.326056 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101859b5-af08-4621-91d4-29745fd3c1a6 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:15.326974 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101859b5-af08-4621-91d4-29745fd3c1a6 None admin] RBAC: Adding query filter params (name=image) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:15.327441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101859b5-af08-4621-91d4-29745fd3c1a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:15.328937 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101859b5-af08-4621-91d4-29745fd3c1a6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:15.334369 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 525/1058] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:50:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:15.339648 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101859b5-af08-4621-91d4-29745fd3c1a6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:15.419298 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033e250-0569-46de-9bc3-40a6ffd8879a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:15.436891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101859b5-af08-4621-91d4-29745fd3c1a6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:15.454576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafda90-c6cf-4383-8db2-7891e4d53502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:15.455257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafda90-c6cf-4383-8db2-7891e4d53502 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:15.455536 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafda90-c6cf-4383-8db2-7891e4d53502 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:15.455808 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafda90-c6cf-4383-8db2-7891e4d53502 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:15.457048 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffafda90-c6cf-4383-8db2-7891e4d53502 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:15.457351 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffafda90-c6cf-4383-8db2-7891e4d53502 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:15.458907 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffafda90-c6cf-4383-8db2-7891e4d53502 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:15.475688 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 534/1059] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:50:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:15.476543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafda90-c6cf-4383-8db2-7891e4d53502 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:15.478001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d30c0-575a-4be6-9631-db79a7dde189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:15.478699 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d30c0-575a-4be6-9631-db79a7dde189 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:15.478976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d30c0-575a-4be6-9631-db79a7dde189 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:15.479250 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d30c0-575a-4be6-9631-db79a7dde189 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:15.479952 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d30c0-575a-4be6-9631-db79a7dde189 None admin] RBAC: Adding query filter params (type=image) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:15.480249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d30c0-575a-4be6-9631-db79a7dde189 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:15.481605 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d30c0-575a-4be6-9631-db79a7dde189 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:15.491509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 526/1060] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:50:15 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:15.492413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d30c0-575a-4be6-9631-db79a7dde189 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:15.576368 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafda90-c6cf-4383-8db2-7891e4d53502 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:15.588419 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d30c0-575a-4be6-9631-db79a7dde189 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:15.609719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c35540-c656-47d4-b9ff-7eb97bd5337e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:15.610537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c35540-c656-47d4-b9ff-7eb97bd5337e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:15.610816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c35540-c656-47d4-b9ff-7eb97bd5337e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:15.611089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c35540-c656-47d4-b9ff-7eb97bd5337e None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:15.617761 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c35540-c656-47d4-b9ff-7eb97bd5337e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:15.619226 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c35540-c656-47d4-b9ff-7eb97bd5337e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:15.624812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91fb40d-deb5-4be4-a1fa-1add03be2dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:15.624812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fb40d-deb5-4be4-a1fa-1add03be2dbf None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:15.624812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fb40d-deb5-4be4-a1fa-1add03be2dbf None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:15.624812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fb40d-deb5-4be4-a1fa-1add03be2dbf None admin] PATH_INFO: `/v3/endpoints` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:15.625178 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91fb40d-deb5-4be4-a1fa-1add03be2dbf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:15.625178 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91fb40d-deb5-4be4-a1fa-1add03be2dbf None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:15.633778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-82c35540-c656-47d4-b9ff-7eb97bd5337e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:50:15.633778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:15.633778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:15.633778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:15.633778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:15.633778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:15.633778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:15.633778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:15.633778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:15.633778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:15.633778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:15.633778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:15.633778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:50:15.633778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:50:15.633778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:50:15.634890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:50:15.634890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:15.634890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:15.634890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:15.634890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:15.634890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:15.634890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:15.634890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:15.634890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:50:15.634890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:15.634890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:15.634890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:15.634890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:15.634890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:15.634890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:15.634890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:15.634890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:15.635988 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:50:15.635988 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:50:15.635988 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:50:15.635988 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:50:15.635988 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:50:15.635988 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:50:15.635988 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:50:15.635988 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:15.635988 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 535/1061] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:50:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:50:15.636507 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700ce1a-0a27-421d-b29f-789e43f39490 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:15.636507 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700ce1a-0a27-421d-b29f-789e43f39490 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:15.641716 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700ce1a-0a27-421d-b29f-789e43f39490 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:15.641716 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 536/1062] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:50:15.642825 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700ce1a-0a27-421d-b29f-789e43f39490 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:15.647751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 527/1063] 10.222.0.23 () {60 vars in 1144 bytes} [Mon May 9 22:50:15 2022] POST /identity/v3/endpoints => generated 327 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 22:50:15.649005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ec1ce-06fd-4079-8cfe-b1ba4e327f88 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:15.649293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ec1ce-06fd-4079-8cfe-b1ba4e327f88 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:15.649568 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ec1ce-06fd-4079-8cfe-b1ba4e327f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:15.667233 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-fd7ec1ce-06fd-4079-8cfe-b1ba4e327f88 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:15.668493 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-fd7ec1ce-06fd-4079-8cfe-b1ba4e327f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:15.746812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 528/1064] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:50:15.756135 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700ce1a-0a27-421d-b29f-789e43f39490 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:15.771165 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ec1ce-06fd-4079-8cfe-b1ba4e327f88 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:15.793357 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716b441-4845-4423-9c88-bd54394501a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:15.794029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716b441-4845-4423-9c88-bd54394501a8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:15.794359 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716b441-4845-4423-9c88-bd54394501a8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:15.794634 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716b441-4845-4423-9c88-bd54394501a8 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:15.795430 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716b441-4845-4423-9c88-bd54394501a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:15.795742 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716b441-4845-4423-9c88-bd54394501a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:15.797210 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716b441-4845-4423-9c88-bd54394501a8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:15.814427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 537/1065] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:50:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:15.816676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716b441-4845-4423-9c88-bd54394501a8 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:15.872910 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ec1ce-06fd-4079-8cfe-b1ba4e327f88 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:15.906752 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb71c33-5757-4df4-9d3a-56c5f258249a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:15.907536 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb71c33-5757-4df4-9d3a-56c5f258249a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:15.907809 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb71c33-5757-4df4-9d3a-56c5f258249a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:15.908075 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb71c33-5757-4df4-9d3a-56c5f258249a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:15.918244 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb71c33-5757-4df4-9d3a-56c5f258249a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:50:15.921077 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716b441-4845-4423-9c88-bd54394501a8 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:15.926091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-9fb71c33-5757-4df4-9d3a-56c5f258249a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:50:15.926091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:15.926091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:15.926091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:15.926091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:15.926091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:15.926091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:15.926091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:15.926091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:15.926091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:15.926091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:15.926091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:15.926091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:50:15.926091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:50:15.926091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:50:15.926091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:50:15.927055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:50:15.927055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:50:15.927055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:50:15.927055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:50:15.927055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:15.927055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:15.927055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:15.927055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:15.927055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:15.927055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:15.927055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:15.927055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:50:15.927055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:15.927055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:15.927055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:15.927055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:15.928514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:15.928514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:15.928514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:15.928514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:15.928514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:50:15.928514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:50:15.928514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:50:15.928514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:50:15.928514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:50:15.928514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:50:15.928514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:50:15.928514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:15.928514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 529/1066] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:50:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:50:15.933116 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb71c33-5757-4df4-9d3a-56c5f258249a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:15.957420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1823a-0623-453b-a169-ccfa2c6a4ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:15.958344 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1823a-0623-453b-a169-ccfa2c6a4ff1 None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:15.958629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1823a-0623-453b-a169-ccfa2c6a4ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:15.958905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1823a-0623-453b-a169-ccfa2c6a4ff1 None admin] PATH_INFO: `/v3/projects/0500e378ea374ca692e230bbd8781318/users/02fcd7cc86134ce488c7dd75dfb174b8/roles/8b7c99010273443e88d3e81ce6924349` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:15.961166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb1823a-0623-453b-a169-ccfa2c6a4ff1 None admin] RBAC: Authorizing `identity:create_grant(project_id=0500e378ea374ca692e230bbd8781318, user_id=02fcd7cc86134ce488c7dd75dfb174b8, role_id=8b7c99010273443e88d3e81ce6924349)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:15.962636 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb1823a-0623-453b-a169-ccfa2c6a4ff1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:15.976012 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 538/1067] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:50:15 2022] PUT /identity/v3/projects/0500e378ea374ca692e230bbd8781318/users/02fcd7cc86134ce488c7dd75dfb174b8/roles/8b7c99010273443e88d3e81ce6924349 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 22:50:16.039542 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb71c33-5757-4df4-9d3a-56c5f258249a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:16.075515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305518b-0c6b-449c-b77d-1fb68e3cea3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:16.076602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305518b-0c6b-449c-b77d-1fb68e3cea3d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:16.077114 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305518b-0c6b-449c-b77d-1fb68e3cea3d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:16.077597 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305518b-0c6b-449c-b77d-1fb68e3cea3d None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:16.078608 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305518b-0c6b-449c-b77d-1fb68e3cea3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:16.079127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305518b-0c6b-449c-b77d-1fb68e3cea3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:16.080711 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305518b-0c6b-449c-b77d-1fb68e3cea3d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:16.095105 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 530/1068] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:50:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:16.096701 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1823a-0623-453b-a169-ccfa2c6a4ff1 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:16.199596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1823a-0623-453b-a169-ccfa2c6a4ff1 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:16.233546 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4e829-f53d-4a54-9952-9349bf2fd4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:16.235082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4e829-f53d-4a54-9952-9349bf2fd4b8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:16.235588 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4e829-f53d-4a54-9952-9349bf2fd4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:16.236066 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4e829-f53d-4a54-9952-9349bf2fd4b8 None admin] PATH_INFO: `/v3/users/8035f81263364841a41f2638b1a8e55c` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:16.250231 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4e829-f53d-4a54-9952-9349bf2fd4b8 None admin] RBAC: Authorizing `identity:get_user(user_id=8035f81263364841a41f2638b1a8e55c)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:16.251970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4e829-f53d-4a54-9952-9349bf2fd4b8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:16.254563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 539/1069] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:50:16 2022] GET /identity/v3/users/8035f81263364841a41f2638b1a8e55c => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:16.260338 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305518b-0c6b-449c-b77d-1fb68e3cea3d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:16.355874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305518b-0c6b-449c-b77d-1fb68e3cea3d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:16.388948 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2d5e6-13d6-4fc9-ba11-7094b72511f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:16.389595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2d5e6-13d6-4fc9-ba11-7094b72511f4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:16.389875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2d5e6-13d6-4fc9-ba11-7094b72511f4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:16.390177 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2d5e6-13d6-4fc9-ba11-7094b72511f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:16.454547 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba2d5e6-13d6-4fc9-ba11-7094b72511f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:16.456000 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba2d5e6-13d6-4fc9-ba11-7094b72511f4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:16.560069 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 531/1070] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:50:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 22:50:16.561546 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4e829-f53d-4a54-9952-9349bf2fd4b8 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:16.661463 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4e829-f53d-4a54-9952-9349bf2fd4b8 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:16.693455 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b6af5-366c-4f83-8686-1269994f5e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:16.694331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b6af5-366c-4f83-8686-1269994f5e96 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:16.694608 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b6af5-366c-4f83-8686-1269994f5e96 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:16.694884 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b6af5-366c-4f83-8686-1269994f5e96 None admin] PATH_INFO: `/v3/projects/0ab5cc8d0bcd47f182e390bfe3867efa` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:16.696035 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b6af5-366c-4f83-8686-1269994f5e96 None admin] RBAC: Authorizing `identity:get_project(project_id=0ab5cc8d0bcd47f182e390bfe3867efa)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:16.697449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b6af5-366c-4f83-8686-1269994f5e96 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:16.704273 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 540/1071] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:50:16 2022] GET /identity/v3/projects/0ab5cc8d0bcd47f182e390bfe3867efa => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:16.705245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2d5e6-13d6-4fc9-ba11-7094b72511f4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:16.801181 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2d5e6-13d6-4fc9-ba11-7094b72511f4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:16.834716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab26b33-dfae-4bb0-b1c9-b22e4c2b1e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:16.835365 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab26b33-dfae-4bb0-b1c9-b22e4c2b1e7b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:16.835641 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab26b33-dfae-4bb0-b1c9-b22e4c2b1e7b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:16.835909 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab26b33-dfae-4bb0-b1c9-b22e4c2b1e7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:16.836720 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab26b33-dfae-4bb0-b1c9-b22e4c2b1e7b None admin] RBAC: Adding query filter params (role.id=bda550492f4f4c5c83f52fa89fc18a7d, scope.project.id=0ab5cc8d0bcd47f182e390bfe3867efa, user.id=8035f81263364841a41f2638b1a8e55c) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:16.837021 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab26b33-dfae-4bb0-b1c9-b22e4c2b1e7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:16.838460 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab26b33-dfae-4bb0-b1c9-b22e4c2b1e7b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:16.850197 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 532/1072] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:50:16 2022] GET /identity/v3/role_assignments?user.id=8035f81263364841a41f2638b1a8e55c&scope.project.id=0ab5cc8d0bcd47f182e390bfe3867efa&role.id=bda550492f4f4c5c83f52fa89fc18a7d => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:17.915769 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd060e4-bacd-4765-bd8d-c852655f93a9 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:17.916277 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd060e4-bacd-4765-bd8d-c852655f93a9 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:17.916277 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd060e4-bacd-4765-bd8d-c852655f93a9 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:17.917644 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 541/1073] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:50:17.923845 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98e1b2-98d8-4e08-8615-c81ec7984fd2 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:17.924334 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98e1b2-98d8-4e08-8615-c81ec7984fd2 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:17.924410 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98e1b2-98d8-4e08-8615-c81ec7984fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:17.941851 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-be98e1b2-98d8-4e08-8615-c81ec7984fd2 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:17.943047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-be98e1b2-98d8-4e08-8615-c81ec7984fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:18.007844 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 533/1074] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:50:18.031826 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd060e4-bacd-4765-bd8d-c852655f93a9 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:18.135339 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd060e4-bacd-4765-bd8d-c852655f93a9 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:18.168118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7420e56-c7f8-452e-9823-f3071ec55026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:18.168688 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7420e56-c7f8-452e-9823-f3071ec55026 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:18.168970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7420e56-c7f8-452e-9823-f3071ec55026 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:18.169244 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7420e56-c7f8-452e-9823-f3071ec55026 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:18.200753 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047840c3-6a68-49b4-befd-53836803ecec None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:18.201036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047840c3-6a68-49b4-befd-53836803ecec None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:18.201313 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047840c3-6a68-49b4-befd-53836803ecec None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:18.202702 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 534/1075] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:50:18.208199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a3162-a411-4b3b-b92c-5f7d148e3c66 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:18.208480 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a3162-a411-4b3b-b92c-5f7d148e3c66 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:18.208752 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a3162-a411-4b3b-b92c-5f7d148e3c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:18.225351 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b79a3162-a411-4b3b-b92c-5f7d148e3c66 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:18.226573 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b79a3162-a411-4b3b-b92c-5f7d148e3c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:18.239531 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7420e56-c7f8-452e-9823-f3071ec55026 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:18.240991 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7420e56-c7f8-452e-9823-f3071ec55026 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:18.291194 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 535/1076] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 22:50:18.314070 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a3162-a411-4b3b-b92c-5f7d148e3c66 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:18.335464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 542/1077] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:50:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 22:50:18.341558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7420e56-c7f8-452e-9823-f3071ec55026 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:18.415658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a3162-a411-4b3b-b92c-5f7d148e3c66 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:18.441832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7420e56-c7f8-452e-9823-f3071ec55026 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:18.448083 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0dc97b-c606-4030-80e7-903ae1114f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:18.448886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0dc97b-c606-4030-80e7-903ae1114f28 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:18.449160 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0dc97b-c606-4030-80e7-903ae1114f28 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:18.449426 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0dc97b-c606-4030-80e7-903ae1114f28 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:18.459768 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0dc97b-c606-4030-80e7-903ae1114f28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:50:18.467716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-df0dc97b-c606-4030-80e7-903ae1114f28 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 22:50:18.467716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:18.467716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:18.467716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:18.467716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:18.467716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:18.467716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:18.467716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:18.467716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:18.467716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:18.467716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:18.467716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:18.467716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:50:18.467716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:50:18.467716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:50:18.467716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:50:18.468749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:50:18.468749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:50:18.468749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:50:18.468749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:50:18.468749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:18.468749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:18.468749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:18.468749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:18.468749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:18.468749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:18.468749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:18.468749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:50:18.468749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:18.468749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:18.468749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:18.468749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:18.475349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:18.475349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:18.475349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:18.475349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:18.475349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:50:18.475349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:50:18.475349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:50:18.475349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:50:18.475349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:50:18.475349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:50:18.475349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 22:50:18.475349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:18.475349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 536/1078] 10.222.0.23 () {56 vars in 1120 bytes} [Mon May 9 22:50:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:50:18.475349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0dc97b-c606-4030-80e7-903ae1114f28 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:18.478003 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90fd3b-fbe2-468e-afef-88d5a85f1b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:18.478835 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90fd3b-fbe2-468e-afef-88d5a85f1b97 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:18.479115 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90fd3b-fbe2-468e-afef-88d5a85f1b97 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:18.479390 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90fd3b-fbe2-468e-afef-88d5a85f1b97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:18.489968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90fd3b-fbe2-468e-afef-88d5a85f1b97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:18.491439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90fd3b-fbe2-468e-afef-88d5a85f1b97 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:18.509141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-4b90fd3b-fbe2-468e-afef-88d5a85f1b97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:50:18.509141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:18.509141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:18.509141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:18.509141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:18.509141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:18.509141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:18.509141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:18.509141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:18.509141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:18.509141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:18.509141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:18.509141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:50:18.509141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:50:18.509141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:50:18.510300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:50:18.510300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:18.510300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:18.510300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:18.510300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:18.510300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:18.510300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:18.510300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:18.510300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:50:18.510300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:18.510300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:18.510300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:18.510300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:18.510300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:18.510300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:18.510300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:18.510300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:18.511244 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:50:18.511244 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:50:18.511244 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:50:18.511244 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:18.511244 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 543/1079] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:50:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:50:18.516085 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90fd3b-fbe2-468e-afef-88d5a85f1b97 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:18.574746 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0dc97b-c606-4030-80e7-903ae1114f28 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:18.608164 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97170d15-efb8-46ba-b7a7-2efcc71265de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:18.608833 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97170d15-efb8-46ba-b7a7-2efcc71265de None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:18.609420 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97170d15-efb8-46ba-b7a7-2efcc71265de None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:18.609705 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97170d15-efb8-46ba-b7a7-2efcc71265de None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:18.610487 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97170d15-efb8-46ba-b7a7-2efcc71265de None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:18.610777 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97170d15-efb8-46ba-b7a7-2efcc71265de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:18.612136 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97170d15-efb8-46ba-b7a7-2efcc71265de None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:18.612501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90fd3b-fbe2-468e-afef-88d5a85f1b97 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:18.625188 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 537/1080] 10.222.0.23 () {56 vars in 1115 bytes} [Mon May 9 22:50:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:18.626716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97170d15-efb8-46ba-b7a7-2efcc71265de None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:18.648961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415423f8-0fda-4b33-b672-137cebf56563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:18.649632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415423f8-0fda-4b33-b672-137cebf56563 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:18.649922 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415423f8-0fda-4b33-b672-137cebf56563 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:18.650220 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415423f8-0fda-4b33-b672-137cebf56563 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:18.650940 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415423f8-0fda-4b33-b672-137cebf56563 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:18.651243 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415423f8-0fda-4b33-b672-137cebf56563 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:18.652668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415423f8-0fda-4b33-b672-137cebf56563 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:18.666716 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 544/1081] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:18.739873 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97170d15-efb8-46ba-b7a7-2efcc71265de None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:18.777043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea9ff2-5fd1-48b1-9a3f-7542a6ff93f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:18.777849 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea9ff2-5fd1-48b1-9a3f-7542a6ff93f4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:18.778141 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea9ff2-5fd1-48b1-9a3f-7542a6ff93f4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:18.778440 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea9ff2-5fd1-48b1-9a3f-7542a6ff93f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:18.790414 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea9ff2-5fd1-48b1-9a3f-7542a6ff93f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:18.791886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea9ff2-5fd1-48b1-9a3f-7542a6ff93f4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:18.810109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-22ea9ff2-5fd1-48b1-9a3f-7542a6ff93f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:50:18.810109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:18.810109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:18.810109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:18.810109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:18.810109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:18.810109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:18.810109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:18.810109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:18.810109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:18.810109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:18.810109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:18.810109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:50:18.810109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:50:18.810109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:50:18.811112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:50:18.811112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:18.811112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:18.811112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:18.811112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:18.811112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:18.811112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:18.811112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:18.811112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:50:18.811112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:18.811112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:18.811112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:18.811112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:18.811112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:18.811112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:18.811112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:18.811112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:18.812036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:50:18.812036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:50:18.812036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:50:18.812036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:18.814676 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 538/1082] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:50:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:50:18.817965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415423f8-0fda-4b33-b672-137cebf56563 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:18.925028 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415423f8-0fda-4b33-b672-137cebf56563 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:18.960486 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84981bb-554a-484f-babb-de44752ae2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:18.961163 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84981bb-554a-484f-babb-de44752ae2fe None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:18.961446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84981bb-554a-484f-babb-de44752ae2fe None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:18.961723 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84981bb-554a-484f-babb-de44752ae2fe None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:18.962503 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84981bb-554a-484f-babb-de44752ae2fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:18.962815 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84981bb-554a-484f-babb-de44752ae2fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:18.964234 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84981bb-554a-484f-babb-de44752ae2fe None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:18.977209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 545/1083] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:18.983127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea9ff2-5fd1-48b1-9a3f-7542a6ff93f4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:19.046687 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055750c-04f3-4d37-9b2c-82d1220df9a8 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:19.046974 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055750c-04f3-4d37-9b2c-82d1220df9a8 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:19.047252 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055750c-04f3-4d37-9b2c-82d1220df9a8 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:19.049812 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 546/1084] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:50:19.055087 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940ea0d-b7bb-456b-972c-a7156af2cb0f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:19.055368 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940ea0d-b7bb-456b-972c-a7156af2cb0f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:19.055637 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940ea0d-b7bb-456b-972c-a7156af2cb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:19.071967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3940ea0d-b7bb-456b-972c-a7156af2cb0f None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:19.073145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3940ea0d-b7bb-456b-972c-a7156af2cb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:19.084669 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea9ff2-5fd1-48b1-9a3f-7542a6ff93f4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:19.117473 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fbe67-4032-4806-860e-71fef2f727b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:19.118435 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fbe67-4032-4806-860e-71fef2f727b5 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:19.118733 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fbe67-4032-4806-860e-71fef2f727b5 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:19.118999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fbe67-4032-4806-860e-71fef2f727b5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:19.126387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259fbe67-4032-4806-860e-71fef2f727b5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:19.127818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259fbe67-4032-4806-860e-71fef2f727b5 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:19.136969 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 547/1085] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 22:50:19.141953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-259fbe67-4032-4806-860e-71fef2f727b5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 22:50:19.141953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:19.141953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:19.141953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:19.141953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:19.141953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:19.141953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:19.141953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:19.141953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:19.141953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:19.141953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:19.141953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:19.141953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:50:19.141953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:50:19.141953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:50:19.141953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:50:19.143130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:19.143130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:19.143130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:50:19.143130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:50:19.143130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:50:19.143130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:50:19.143130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:19.143130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:19.143130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:19.143130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:19.143130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:19.143130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:50:19.143130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:19.143130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:19.143130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:19.143130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:19.143130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:19.143130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:19.144816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:19.144816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:19.144816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:50:19.144816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:50:19.144816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:50:19.144816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:50:19.144816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:50:19.144816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:50:19.144816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 22:50:19.144816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:19.144816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 539/1086] 10.222.0.23 () {56 vars in 1093 bytes} [Mon May 9 22:50:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:50:19.149490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940ea0d-b7bb-456b-972c-a7156af2cb0f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:19.160616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fbe67-4032-4806-860e-71fef2f727b5 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:19.248132 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940ea0d-b7bb-456b-972c-a7156af2cb0f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:19.259624 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fbe67-4032-4806-860e-71fef2f727b5 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:19.281003 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00d6ff-78c7-481d-a9b1-a4ac39dc7105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:19.281669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00d6ff-78c7-481d-a9b1-a4ac39dc7105 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:19.281950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00d6ff-78c7-481d-a9b1-a4ac39dc7105 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:19.282261 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00d6ff-78c7-481d-a9b1-a4ac39dc7105 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:19.283093 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d00d6ff-78c7-481d-a9b1-a4ac39dc7105 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:19.283401 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d00d6ff-78c7-481d-a9b1-a4ac39dc7105 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:19.285412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d00d6ff-78c7-481d-a9b1-a4ac39dc7105 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:19.299435 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339236c-ea06-41d9-a136-97173d3ccc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:19.300199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339236c-ea06-41d9-a136-97173d3ccc83 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:19.300479 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339236c-ea06-41d9-a136-97173d3ccc83 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:19.300753 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339236c-ea06-41d9-a136-97173d3ccc83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:19.303481 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 548/1087] 10.222.0.23 () {56 vars in 1133 bytes} [Mon May 9 22:50:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:19.309104 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00d6ff-78c7-481d-a9b1-a4ac39dc7105 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:19.311031 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3339236c-ea06-41d9-a136-97173d3ccc83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:50:19.319274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-3339236c-ea06-41d9-a136-97173d3ccc83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:50:19.319274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:19.319274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:19.319274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:19.319274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:19.319274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:19.319274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:19.319274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:19.319274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:19.319274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:19.319274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:19.319274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:19.319274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:50:19.319274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:50:19.319274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:50:19.319274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:50:19.320345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:50:19.320345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:50:19.320345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:50:19.320345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:50:19.320345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:19.320345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:19.320345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:19.320345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:19.320345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:19.320345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:19.320345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:19.320345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:50:19.320345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:19.320345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:19.320345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:19.320345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:19.321388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:19.321388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:19.321388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:19.321388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:19.321388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:50:19.321388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:50:19.321388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:50:19.321388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:50:19.321388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:50:19.321388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:50:19.321388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:50:19.321388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:19.321388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 540/1088] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:50:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:50:19.326350 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339236c-ea06-41d9-a136-97173d3ccc83 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:19.437180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339236c-ea06-41d9-a136-97173d3ccc83 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:19.445157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00d6ff-78c7-481d-a9b1-a4ac39dc7105 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:19.488130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c758ed5-31e0-465c-a152-406c486ac46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:19.489186 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c758ed5-31e0-465c-a152-406c486ac46b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:19.489635 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c758ed5-31e0-465c-a152-406c486ac46b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:19.490087 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c758ed5-31e0-465c-a152-406c486ac46b None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:19.491243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c758ed5-31e0-465c-a152-406c486ac46b None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:19.491748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c758ed5-31e0-465c-a152-406c486ac46b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:19.494145 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c758ed5-31e0-465c-a152-406c486ac46b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:19.509719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 541/1089] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:50:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:19.509909 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff2950-800b-4ecf-aa55-b4fe105b16bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:19.509909 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff2950-800b-4ecf-aa55-b4fe105b16bb None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:19.509909 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff2950-800b-4ecf-aa55-b4fe105b16bb None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:19.509909 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff2950-800b-4ecf-aa55-b4fe105b16bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:19.511053 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c758ed5-31e0-465c-a152-406c486ac46b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:19.577721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff2950-800b-4ecf-aa55-b4fe105b16bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:19.579101 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff2950-800b-4ecf-aa55-b4fe105b16bb None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:19.611783 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c758ed5-31e0-465c-a152-406c486ac46b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:19.647501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666f7f3-c73c-4102-9f96-cc388789747e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:19.648373 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666f7f3-c73c-4102-9f96-cc388789747e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:19.648648 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666f7f3-c73c-4102-9f96-cc388789747e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:19.648919 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666f7f3-c73c-4102-9f96-cc388789747e None admin] PATH_INFO: `/v3/users/8035f81263364841a41f2638b1a8e55c` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:19.650160 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666f7f3-c73c-4102-9f96-cc388789747e None admin] RBAC: Authorizing `identity:get_user(user_id=8035f81263364841a41f2638b1a8e55c)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:19.651592 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666f7f3-c73c-4102-9f96-cc388789747e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:19.653229 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 542/1090] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:50:19 2022] GET /identity/v3/users/8035f81263364841a41f2638b1a8e55c => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:19.658749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666f7f3-c73c-4102-9f96-cc388789747e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:19.699187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 549/1091] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:50:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 22:50:19.700296 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff2950-800b-4ecf-aa55-b4fe105b16bb None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:19.760789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666f7f3-c73c-4102-9f96-cc388789747e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:19.793362 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499696e-475b-4460-b9c5-f7bbc901e26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:19.795080 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499696e-475b-4460-b9c5-f7bbc901e26c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:19.795080 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499696e-475b-4460-b9c5-f7bbc901e26c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:19.795080 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499696e-475b-4460-b9c5-f7bbc901e26c None admin] PATH_INFO: `/v3/projects/0ab5cc8d0bcd47f182e390bfe3867efa` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:19.795927 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9499696e-475b-4460-b9c5-f7bbc901e26c None admin] RBAC: Authorizing `identity:get_project(project_id=0ab5cc8d0bcd47f182e390bfe3867efa)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:19.796871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff2950-800b-4ecf-aa55-b4fe105b16bb None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:19.797346 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9499696e-475b-4460-b9c5-f7bbc901e26c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:19.799232 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 543/1092] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:50:19 2022] GET /identity/v3/projects/0ab5cc8d0bcd47f182e390bfe3867efa => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:19.804854 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499696e-475b-4460-b9c5-f7bbc901e26c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:19.832290 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c8052-02af-4cf7-bcbb-03e4072a714c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:19.832988 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c8052-02af-4cf7-bcbb-03e4072a714c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:19.833264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c8052-02af-4cf7-bcbb-03e4072a714c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:19.833531 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c8052-02af-4cf7-bcbb-03e4072a714c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:19.844081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c8052-02af-4cf7-bcbb-03e4072a714c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:19.845522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c8052-02af-4cf7-bcbb-03e4072a714c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:19.863138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-007c8052-02af-4cf7-bcbb-03e4072a714c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:50:19.863138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:19.863138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:19.863138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:19.863138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:19.863138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:19.863138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:19.863138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:19.863138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:19.863138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:19.863138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:19.863138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:19.863138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:50:19.863138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:50:19.863138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:50:19.864270 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:50:19.864270 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:19.864270 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:19.864270 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:19.864270 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:19.864270 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:19.864270 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:19.864270 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:19.864270 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:50:19.864270 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:19.864270 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:19.864270 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:19.864270 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:19.864270 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:19.864270 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:19.864270 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:19.864270 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:19.865652 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:50:19.865652 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:50:19.865652 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:50:19.865652 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:19.865652 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 550/1093] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:50:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:50:19.869811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c8052-02af-4cf7-bcbb-03e4072a714c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:19.902609 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499696e-475b-4460-b9c5-f7bbc901e26c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:19.934396 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6e9d3-fe75-4728-8f43-fbbc4b3443ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:19.935284 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6e9d3-fe75-4728-8f43-fbbc4b3443ca None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:19.935561 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6e9d3-fe75-4728-8f43-fbbc4b3443ca None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:19.935838 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6e9d3-fe75-4728-8f43-fbbc4b3443ca None admin] PATH_INFO: `/v3/projects/0ab5cc8d0bcd47f182e390bfe3867efa/users/8035f81263364841a41f2638b1a8e55c/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:19.937802 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6e9d3-fe75-4728-8f43-fbbc4b3443ca None admin] RBAC: Authorizing `identity:create_grant(project_id=0ab5cc8d0bcd47f182e390bfe3867efa, user_id=8035f81263364841a41f2638b1a8e55c, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:19.939276 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6e9d3-fe75-4728-8f43-fbbc4b3443ca None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:19.953359 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 544/1094] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:50:19 2022] PUT /identity/v3/projects/0ab5cc8d0bcd47f182e390bfe3867efa/users/8035f81263364841a41f2638b1a8e55c/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 22:50:19.969035 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c8052-02af-4cf7-bcbb-03e4072a714c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:20.010722 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc330632-e384-4857-8d56-4d5b38d75827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:20.011405 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc330632-e384-4857-8d56-4d5b38d75827 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:20.011687 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc330632-e384-4857-8d56-4d5b38d75827 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:20.011961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc330632-e384-4857-8d56-4d5b38d75827 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:20.012703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc330632-e384-4857-8d56-4d5b38d75827 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:20.013009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc330632-e384-4857-8d56-4d5b38d75827 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:20.014447 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc330632-e384-4857-8d56-4d5b38d75827 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:20.026902 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 551/1095] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:20.033115 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6e9d3-fe75-4728-8f43-fbbc4b3443ca None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:20.136616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6e9d3-fe75-4728-8f43-fbbc4b3443ca None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:20.168637 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57729bac-c8ff-4ec7-afed-c12050b90062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:20.169427 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57729bac-c8ff-4ec7-afed-c12050b90062 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:20.169701 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57729bac-c8ff-4ec7-afed-c12050b90062 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:20.169966 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57729bac-c8ff-4ec7-afed-c12050b90062 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:20.176898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57729bac-c8ff-4ec7-afed-c12050b90062 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:20.178344 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57729bac-c8ff-4ec7-afed-c12050b90062 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:20.193127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-57729bac-c8ff-4ec7-afed-c12050b90062 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:50:20.193127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:20.193127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:20.193127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:20.193127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:20.193127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:20.193127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:20.193127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:20.193127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:20.193127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:20.193127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:20.193127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:20.193127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:50:20.193127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:50:20.193127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:50:20.194097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:50:20.194097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:20.194097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:20.194097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:20.194097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:20.194097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:20.194097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:20.194097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:20.194097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:50:20.194097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:20.194097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:20.194097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:20.194097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:20.194097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:20.194097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:20.194097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:20.194097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:20.195081 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:50:20.195081 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:50:20.195081 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:50:20.195081 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:50:20.195081 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:50:20.195081 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:50:20.195081 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:50:20.195081 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:20.195081 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 545/1096] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:50:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:50:20.200054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc330632-e384-4857-8d56-4d5b38d75827 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:20.294910 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc330632-e384-4857-8d56-4d5b38d75827 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:20.327053 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25b5f5-bec2-4346-b495-106a8a75a5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:20.327702 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25b5f5-bec2-4346-b495-106a8a75a5ce None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:20.327978 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25b5f5-bec2-4346-b495-106a8a75a5ce None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:20.328244 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25b5f5-bec2-4346-b495-106a8a75a5ce None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:20.329011 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25b5f5-bec2-4346-b495-106a8a75a5ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:20.329312 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25b5f5-bec2-4346-b495-106a8a75a5ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:20.330745 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25b5f5-bec2-4346-b495-106a8a75a5ce None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:20.342102 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 552/1097] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:50:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:20.348563 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57729bac-c8ff-4ec7-afed-c12050b90062 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:20.441348 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57729bac-c8ff-4ec7-afed-c12050b90062 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:20.473215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515921ad-7b90-494e-b106-dadd758d2f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:20.473852 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515921ad-7b90-494e-b106-dadd758d2f63 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:20.474143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515921ad-7b90-494e-b106-dadd758d2f63 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:20.474411 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515921ad-7b90-494e-b106-dadd758d2f63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:20.475267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515921ad-7b90-494e-b106-dadd758d2f63 None admin] RBAC: Adding query filter params (role.id=8b7c99010273443e88d3e81ce6924349, scope.project.id=0500e378ea374ca692e230bbd8781318, user.id=02fcd7cc86134ce488c7dd75dfb174b8) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:20.475572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515921ad-7b90-494e-b106-dadd758d2f63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:20.476972 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515921ad-7b90-494e-b106-dadd758d2f63 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:20.482491 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 546/1098] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:50:20 2022] GET /identity/v3/role_assignments?user.id=02fcd7cc86134ce488c7dd75dfb174b8&scope.project.id=0500e378ea374ca692e230bbd8781318&role.id=8b7c99010273443e88d3e81ce6924349 => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:22.183367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e0711-f0e6-45bb-81ef-5aee7397a524 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:22.183899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e0711-f0e6-45bb-81ef-5aee7397a524 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:22.183991 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e0711-f0e6-45bb-81ef-5aee7397a524 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:22.185312 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 553/1099] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:50:22.190574 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b12f74-65d2-48b8-8ea9-1ce8ca608473 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:22.191009 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b12f74-65d2-48b8-8ea9-1ce8ca608473 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:22.191103 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b12f74-65d2-48b8-8ea9-1ce8ca608473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:22.207739 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d9b12f74-65d2-48b8-8ea9-1ce8ca608473 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:22.208901 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d9b12f74-65d2-48b8-8ea9-1ce8ca608473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:22.277290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 547/1100] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 22:50:22.301118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e0711-f0e6-45bb-81ef-5aee7397a524 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:22.396815 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e0711-f0e6-45bb-81ef-5aee7397a524 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:22.428263 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd73b0-e38f-4219-ad8f-6c06fa9a7e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:22.429034 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd73b0-e38f-4219-ad8f-6c06fa9a7e8c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:22.429310 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd73b0-e38f-4219-ad8f-6c06fa9a7e8c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:22.429580 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd73b0-e38f-4219-ad8f-6c06fa9a7e8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:22.439109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd73b0-e38f-4219-ad8f-6c06fa9a7e8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:50:22.446605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-12cd73b0-e38f-4219-ad8f-6c06fa9a7e8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:50:22.446605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:22.446605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:22.446605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:22.446605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:22.446605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:22.446605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:22.446605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:22.446605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:22.446605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:22.446605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:22.446605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:22.446605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:50:22.446605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:50:22.446605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:50:22.446605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:50:22.447686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:50:22.447686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:50:22.447686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:50:22.447686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:50:22.447686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:22.447686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:22.447686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:22.447686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:22.447686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:22.447686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:22.447686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:22.447686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:50:22.447686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:22.447686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:22.447686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:22.447686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:22.448633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:22.448633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:22.448633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:22.448633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:22.448633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:50:22.448633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:50:22.448633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:50:22.448633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:50:22.448633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:50:22.448633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:50:22.448633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:50:22.448633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:22.448633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 554/1101] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:50:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:50:22.453743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b12f74-65d2-48b8-8ea9-1ce8ca608473 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:22.545112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b12f74-65d2-48b8-8ea9-1ce8ca608473 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:22.578926 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f75c04-080d-4c8f-b0fc-f8e275995e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:22.579566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f75c04-080d-4c8f-b0fc-f8e275995e28 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:22.579839 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f75c04-080d-4c8f-b0fc-f8e275995e28 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:22.580107 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f75c04-080d-4c8f-b0fc-f8e275995e28 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:22.580824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f75c04-080d-4c8f-b0fc-f8e275995e28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:22.581665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f75c04-080d-4c8f-b0fc-f8e275995e28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:22.583041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f75c04-080d-4c8f-b0fc-f8e275995e28 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:22.590524 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 548/1102] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:50:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:22.596113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd73b0-e38f-4219-ad8f-6c06fa9a7e8c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:22.689898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd73b0-e38f-4219-ad8f-6c06fa9a7e8c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:22.721489 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e61499-a308-4b59-a604-19ce6a0c6853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:22.722386 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e61499-a308-4b59-a604-19ce6a0c6853 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:22.722664 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e61499-a308-4b59-a604-19ce6a0c6853 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:22.722934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e61499-a308-4b59-a604-19ce6a0c6853 None admin] PATH_INFO: `/v3/users/8035f81263364841a41f2638b1a8e55c` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:22.724560 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e61499-a308-4b59-a604-19ce6a0c6853 None admin] RBAC: Authorizing `identity:get_user(user_id=8035f81263364841a41f2638b1a8e55c)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:22.726593 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e61499-a308-4b59-a604-19ce6a0c6853 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:22.728249 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 555/1103] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:50:22 2022] GET /identity/v3/users/8035f81263364841a41f2638b1a8e55c => generated 285 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:22.734522 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f75c04-080d-4c8f-b0fc-f8e275995e28 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:22.826623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f75c04-080d-4c8f-b0fc-f8e275995e28 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:22.857302 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a8a86-bfb3-4cd4-b619-e322d27c76fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:22.857928 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a8a86-bfb3-4cd4-b619-e322d27c76fb None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:22.858229 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a8a86-bfb3-4cd4-b619-e322d27c76fb None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:22.858500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a8a86-bfb3-4cd4-b619-e322d27c76fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:22.919729 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a8a86-bfb3-4cd4-b619-e322d27c76fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:22.921141 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a8a86-bfb3-4cd4-b619-e322d27c76fb None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:23.011004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 549/1104] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:50:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 22:50:23.016733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e61499-a308-4b59-a604-19ce6a0c6853 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:23.111293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e61499-a308-4b59-a604-19ce6a0c6853 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:23.142308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb9681f-23ca-4d63-b4f8-da062431b166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:23.143142 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9681f-23ca-4d63-b4f8-da062431b166 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:23.143422 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9681f-23ca-4d63-b4f8-da062431b166 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:23.144221 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9681f-23ca-4d63-b4f8-da062431b166 None admin] PATH_INFO: `/v3/projects/0ab5cc8d0bcd47f182e390bfe3867efa` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:23.145372 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb9681f-23ca-4d63-b4f8-da062431b166 None admin] RBAC: Authorizing `identity:get_project(project_id=0ab5cc8d0bcd47f182e390bfe3867efa)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:23.146869 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb9681f-23ca-4d63-b4f8-da062431b166 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:23.148631 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 556/1105] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:50:23 2022] GET /identity/v3/projects/0ab5cc8d0bcd47f182e390bfe3867efa => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:23.154433 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a8a86-bfb3-4cd4-b619-e322d27c76fb None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:23.247543 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a8a86-bfb3-4cd4-b619-e322d27c76fb None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:23.278662 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02923b96-1545-4109-8980-415c21467dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:23.279287 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02923b96-1545-4109-8980-415c21467dba None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:23.279558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02923b96-1545-4109-8980-415c21467dba None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:23.279821 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02923b96-1545-4109-8980-415c21467dba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:23.280655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02923b96-1545-4109-8980-415c21467dba None admin] RBAC: Adding query filter params (role.id=bda550492f4f4c5c83f52fa89fc18a7d, scope.project.id=0ab5cc8d0bcd47f182e390bfe3867efa, user.id=8035f81263364841a41f2638b1a8e55c) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:23.280954 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02923b96-1545-4109-8980-415c21467dba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:23.282368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02923b96-1545-4109-8980-415c21467dba None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:23.288399 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 550/1106] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:50:23 2022] GET /identity/v3/role_assignments?user.id=8035f81263364841a41f2638b1a8e55c&scope.project.id=0ab5cc8d0bcd47f182e390bfe3867efa&role.id=bda550492f4f4c5c83f52fa89fc18a7d => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:25.517012 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7469ff-2a0a-4a6e-9658-4fffe1f32ac2 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:25.517543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7469ff-2a0a-4a6e-9658-4fffe1f32ac2 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:25.517543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7469ff-2a0a-4a6e-9658-4fffe1f32ac2 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:25.518976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 557/1107] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:50:25.524135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ebd15-a151-4c68-80f0-cbe15c7cfade None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:25.524555 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ebd15-a151-4c68-80f0-cbe15c7cfade None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:25.524686 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ebd15-a151-4c68-80f0-cbe15c7cfade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:25.540718 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-fe9ebd15-a151-4c68-80f0-cbe15c7cfade None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:25.541873 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-fe9ebd15-a151-4c68-80f0-cbe15c7cfade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:25.604238 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 551/1108] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:50:25.632668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7469ff-2a0a-4a6e-9658-4fffe1f32ac2 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:25.729434 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7469ff-2a0a-4a6e-9658-4fffe1f32ac2 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:25.760730 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0070e98-840f-4267-994b-3456c980b3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:25.761499 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0070e98-840f-4267-994b-3456c980b3ff None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:25.761773 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0070e98-840f-4267-994b-3456c980b3ff None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:25.762042 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0070e98-840f-4267-994b-3456c980b3ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:25.763216 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0070e98-840f-4267-994b-3456c980b3ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:25.764631 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0070e98-840f-4267-994b-3456c980b3ff None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:25.766232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 558/1109] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:50:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:25.772473 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ebd15-a151-4c68-80f0-cbe15c7cfade None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:25.864123 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ebd15-a151-4c68-80f0-cbe15c7cfade None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:25.896141 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9014d3-60be-48c7-a249-15e5b30e5162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:25.896785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9014d3-60be-48c7-a249-15e5b30e5162 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:25.897065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9014d3-60be-48c7-a249-15e5b30e5162 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:25.897334 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9014d3-60be-48c7-a249-15e5b30e5162 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:25.898036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9014d3-60be-48c7-a249-15e5b30e5162 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:25.899473 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9014d3-60be-48c7-a249-15e5b30e5162 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:25.917704 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 552/1110] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 9 22:50:25 2022] POST /identity/v3/users => generated 286 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:50:28.138653 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9af8d-bbe1-4948-bd80-7eb81a413be9 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:28.139238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9af8d-bbe1-4948-bd80-7eb81a413be9 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:28.139238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9af8d-bbe1-4948-bd80-7eb81a413be9 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:28.140577 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 559/1111] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:50:28.146257 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f8d74-bf6d-473c-bafd-b93e6f961ab8 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:28.146702 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f8d74-bf6d-473c-bafd-b93e6f961ab8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:28.146800 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f8d74-bf6d-473c-bafd-b93e6f961ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:28.163220 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-f95f8d74-bf6d-473c-bafd-b93e6f961ab8 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:28.164364 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f95f8d74-bf6d-473c-bafd-b93e6f961ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:28.226132 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 553/1112] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:50:28.248886 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9af8d-bbe1-4948-bd80-7eb81a413be9 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:28.343829 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9af8d-bbe1-4948-bd80-7eb81a413be9 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:28.375354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776cabc-bdff-4f96-aeeb-7884b8c7e9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:28.376193 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776cabc-bdff-4f96-aeeb-7884b8c7e9cd None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:28.376469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776cabc-bdff-4f96-aeeb-7884b8c7e9cd None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:28.376742 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776cabc-bdff-4f96-aeeb-7884b8c7e9cd None admin] PATH_INFO: `/v3/roles/46ddc3cf4b124399a275bc639635e10a` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:28.378242 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776cabc-bdff-4f96-aeeb-7884b8c7e9cd None admin] RBAC: Authorizing `identity:get_role(role_id=46ddc3cf4b124399a275bc639635e10a)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:28.379602 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776cabc-bdff-4f96-aeeb-7884b8c7e9cd None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:28.380936 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 560/1113] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:50:28 2022] GET /identity/v3/roles/46ddc3cf4b124399a275bc639635e10a => generated 223 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:28.386551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f8d74-bf6d-473c-bafd-b93e6f961ab8 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:28.481595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f8d74-bf6d-473c-bafd-b93e6f961ab8 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:28.514447 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340fb292-e082-4a65-a5be-951c6fa11ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:28.515297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fb292-e082-4a65-a5be-951c6fa11ac6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:28.515575 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fb292-e082-4a65-a5be-951c6fa11ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:28.515847 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fb292-e082-4a65-a5be-951c6fa11ac6 None admin] PATH_INFO: `/v3/users/1459bd8dfb294c5d9ffdc11426111737` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:28.528164 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340fb292-e082-4a65-a5be-951c6fa11ac6 None admin] RBAC: Authorizing `identity:get_user(user_id=1459bd8dfb294c5d9ffdc11426111737)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:28.529581 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340fb292-e082-4a65-a5be-951c6fa11ac6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:28.531225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 554/1114] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:50:28 2022] GET /identity/v3/users/1459bd8dfb294c5d9ffdc11426111737 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:28.536928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776cabc-bdff-4f96-aeeb-7884b8c7e9cd None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:28.631846 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776cabc-bdff-4f96-aeeb-7884b8c7e9cd None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:28.665978 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52be1a-5cc8-466f-ab00-cb3dcf6d0309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:28.666642 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52be1a-5cc8-466f-ab00-cb3dcf6d0309 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:28.666923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52be1a-5cc8-466f-ab00-cb3dcf6d0309 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:28.667194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52be1a-5cc8-466f-ab00-cb3dcf6d0309 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:28.731432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f52be1a-5cc8-466f-ab00-cb3dcf6d0309 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:28.733033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f52be1a-5cc8-466f-ab00-cb3dcf6d0309 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:28.822722 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 561/1115] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:50:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 22:50:28.828539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340fb292-e082-4a65-a5be-951c6fa11ac6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:28.922379 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340fb292-e082-4a65-a5be-951c6fa11ac6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:28.955171 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a7852-7507-4583-bc55-67afdcf0087f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:28.956023 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a7852-7507-4583-bc55-67afdcf0087f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:28.956304 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a7852-7507-4583-bc55-67afdcf0087f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:28.956578 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a7852-7507-4583-bc55-67afdcf0087f None admin] PATH_INFO: `/v3/projects/0ab5cc8d0bcd47f182e390bfe3867efa` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:28.957725 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a7852-7507-4583-bc55-67afdcf0087f None admin] RBAC: Authorizing `identity:get_project(project_id=0ab5cc8d0bcd47f182e390bfe3867efa)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:28.959173 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a7852-7507-4583-bc55-67afdcf0087f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:28.960831 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 555/1116] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:50:28 2022] GET /identity/v3/projects/0ab5cc8d0bcd47f182e390bfe3867efa => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:28.966743 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52be1a-5cc8-466f-ab00-cb3dcf6d0309 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:29.066291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52be1a-5cc8-466f-ab00-cb3dcf6d0309 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:29.097177 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f8253-f8cf-4931-8b3b-b698bdf8623d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:29.097808 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f8253-f8cf-4931-8b3b-b698bdf8623d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:29.098083 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f8253-f8cf-4931-8b3b-b698bdf8623d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:29.098378 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f8253-f8cf-4931-8b3b-b698bdf8623d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:29.099187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f8253-f8cf-4931-8b3b-b698bdf8623d None admin] RBAC: Adding query filter params (role.id=46ddc3cf4b124399a275bc639635e10a, scope.project.id=0ab5cc8d0bcd47f182e390bfe3867efa, user.id=1459bd8dfb294c5d9ffdc11426111737) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:29.099491 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f8253-f8cf-4931-8b3b-b698bdf8623d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:29.100901 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f8253-f8cf-4931-8b3b-b698bdf8623d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:29.106770 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 562/1117] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:50:28 2022] GET /identity/v3/role_assignments?user.id=1459bd8dfb294c5d9ffdc11426111737&scope.project.id=0ab5cc8d0bcd47f182e390bfe3867efa&role.id=46ddc3cf4b124399a275bc639635e10a => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:31.264779 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71027e8-98f1-49d2-95d1-508380c2cd83 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:31.265334 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71027e8-98f1-49d2-95d1-508380c2cd83 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:31.265334 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71027e8-98f1-49d2-95d1-508380c2cd83 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:31.266709 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 556/1118] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:50:31.272391 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51cca1d-7fb6-430f-a94d-09965f6a625f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:31.272806 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51cca1d-7fb6-430f-a94d-09965f6a625f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:31.272927 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51cca1d-7fb6-430f-a94d-09965f6a625f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:31.289655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e51cca1d-7fb6-430f-a94d-09965f6a625f None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:31.290849 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e51cca1d-7fb6-430f-a94d-09965f6a625f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:31.352534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 563/1119] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:50:31.374888 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71027e8-98f1-49d2-95d1-508380c2cd83 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:31.468308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71027e8-98f1-49d2-95d1-508380c2cd83 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:31.499665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1cad59-64b4-41b6-9364-d225eca5aef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:31.500486 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1cad59-64b4-41b6-9364-d225eca5aef1 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:31.500756 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1cad59-64b4-41b6-9364-d225eca5aef1 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:31.501022 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1cad59-64b4-41b6-9364-d225eca5aef1 None admin] PATH_INFO: `/v3/roles/46ddc3cf4b124399a275bc639635e10a` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:31.502474 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1cad59-64b4-41b6-9364-d225eca5aef1 None admin] RBAC: Authorizing `identity:get_role(role_id=46ddc3cf4b124399a275bc639635e10a)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:31.504348 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1cad59-64b4-41b6-9364-d225eca5aef1 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:31.505657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 557/1120] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:50:31 2022] GET /identity/v3/roles/46ddc3cf4b124399a275bc639635e10a => generated 223 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:31.511356 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51cca1d-7fb6-430f-a94d-09965f6a625f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:31.602693 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51cca1d-7fb6-430f-a94d-09965f6a625f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:31.634685 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24191c5f-ea27-4ecc-82d1-8c190ce6ae61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:31.635543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24191c5f-ea27-4ecc-82d1-8c190ce6ae61 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:31.635826 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24191c5f-ea27-4ecc-82d1-8c190ce6ae61 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:31.636102 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24191c5f-ea27-4ecc-82d1-8c190ce6ae61 None admin] PATH_INFO: `/v3/users/1459bd8dfb294c5d9ffdc11426111737` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:31.637280 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24191c5f-ea27-4ecc-82d1-8c190ce6ae61 None admin] RBAC: Authorizing `identity:get_user(user_id=1459bd8dfb294c5d9ffdc11426111737)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:31.638739 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24191c5f-ea27-4ecc-82d1-8c190ce6ae61 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:31.640357 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 564/1121] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:50:31 2022] GET /identity/v3/users/1459bd8dfb294c5d9ffdc11426111737 => generated 286 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:31.646070 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1cad59-64b4-41b6-9364-d225eca5aef1 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:31.738012 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1cad59-64b4-41b6-9364-d225eca5aef1 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:31.769520 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d3662-08da-424d-b747-e1899b6b709a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:31.770369 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d3662-08da-424d-b747-e1899b6b709a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:31.770644 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d3662-08da-424d-b747-e1899b6b709a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:31.770911 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d3662-08da-424d-b747-e1899b6b709a None admin] PATH_INFO: `/v3/projects/0ab5cc8d0bcd47f182e390bfe3867efa` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:31.772047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2d3662-08da-424d-b747-e1899b6b709a None admin] RBAC: Authorizing `identity:get_project(project_id=0ab5cc8d0bcd47f182e390bfe3867efa)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:31.773444 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2d3662-08da-424d-b747-e1899b6b709a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:31.775088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 558/1122] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:50:31 2022] GET /identity/v3/projects/0ab5cc8d0bcd47f182e390bfe3867efa => generated 311 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:31.780746 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24191c5f-ea27-4ecc-82d1-8c190ce6ae61 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:31.872458 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24191c5f-ea27-4ecc-82d1-8c190ce6ae61 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:31.905078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036080b-51b2-482a-ae24-d1e471fa5b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:31.905935 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036080b-51b2-482a-ae24-d1e471fa5b3e None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:31.906244 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036080b-51b2-482a-ae24-d1e471fa5b3e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:31.906517 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036080b-51b2-482a-ae24-d1e471fa5b3e None admin] PATH_INFO: `/v3/projects/0ab5cc8d0bcd47f182e390bfe3867efa/users/1459bd8dfb294c5d9ffdc11426111737/roles/46ddc3cf4b124399a275bc639635e10a` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:31.908671 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036080b-51b2-482a-ae24-d1e471fa5b3e None admin] RBAC: Authorizing `identity:create_grant(project_id=0ab5cc8d0bcd47f182e390bfe3867efa, user_id=1459bd8dfb294c5d9ffdc11426111737, role_id=46ddc3cf4b124399a275bc639635e10a)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:31.910128 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036080b-51b2-482a-ae24-d1e471fa5b3e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:31.922042 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 565/1123] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:50:31 2022] PUT /identity/v3/projects/0ab5cc8d0bcd47f182e390bfe3867efa/users/1459bd8dfb294c5d9ffdc11426111737/roles/46ddc3cf4b124399a275bc639635e10a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:50:34.102527 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c9409-35f6-4c9b-92ce-f7f7c22e278b None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:34.103084 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c9409-35f6-4c9b-92ce-f7f7c22e278b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:34.103084 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c9409-35f6-4c9b-92ce-f7f7c22e278b None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:34.104415 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 559/1124] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:50:34.109915 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5dd408-a0de-4241-a5f3-39c3b1216d49 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:34.110391 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5dd408-a0de-4241-a5f3-39c3b1216d49 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:34.110510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5dd408-a0de-4241-a5f3-39c3b1216d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:34.126160 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5f5dd408-a0de-4241-a5f3-39c3b1216d49 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:34.127330 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5f5dd408-a0de-4241-a5f3-39c3b1216d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:34.190068 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 566/1125] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:50:34.217137 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c9409-35f6-4c9b-92ce-f7f7c22e278b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:34.312231 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c9409-35f6-4c9b-92ce-f7f7c22e278b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:34.343758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a55e3-72b7-474d-aeba-c718b177adbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:34.344586 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a55e3-72b7-474d-aeba-c718b177adbe None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:34.344863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a55e3-72b7-474d-aeba-c718b177adbe None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:34.345133 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a55e3-72b7-474d-aeba-c718b177adbe None admin] PATH_INFO: `/v3/roles/46ddc3cf4b124399a275bc639635e10a` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:34.346625 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a55e3-72b7-474d-aeba-c718b177adbe None admin] RBAC: Authorizing `identity:get_role(role_id=46ddc3cf4b124399a275bc639635e10a)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:34.347981 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a55e3-72b7-474d-aeba-c718b177adbe None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:34.349316 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 560/1126] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:50:34 2022] GET /identity/v3/roles/46ddc3cf4b124399a275bc639635e10a => generated 223 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:34.355072 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5dd408-a0de-4241-a5f3-39c3b1216d49 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:34.447829 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5dd408-a0de-4241-a5f3-39c3b1216d49 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:34.482717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb836050-ca94-49c2-ab7e-7fc83327e10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:34.483574 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb836050-ca94-49c2-ab7e-7fc83327e10b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:34.484109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb836050-ca94-49c2-ab7e-7fc83327e10b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:34.484386 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb836050-ca94-49c2-ab7e-7fc83327e10b None admin] PATH_INFO: `/v3/users/1459bd8dfb294c5d9ffdc11426111737` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:34.485558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb836050-ca94-49c2-ab7e-7fc83327e10b None admin] RBAC: Authorizing `identity:get_user(user_id=1459bd8dfb294c5d9ffdc11426111737)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:34.487553 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb836050-ca94-49c2-ab7e-7fc83327e10b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:34.489184 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 567/1127] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:50:34 2022] GET /identity/v3/users/1459bd8dfb294c5d9ffdc11426111737 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:34.495005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a55e3-72b7-474d-aeba-c718b177adbe None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:34.586123 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a55e3-72b7-474d-aeba-c718b177adbe None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:34.618488 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d9576-63e7-49da-b953-3a87a2283a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:34.619050 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d9576-63e7-49da-b953-3a87a2283a28 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:34.619326 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d9576-63e7-49da-b953-3a87a2283a28 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:34.619595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d9576-63e7-49da-b953-3a87a2283a28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:34.681335 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d9576-63e7-49da-b953-3a87a2283a28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:34.682792 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d9576-63e7-49da-b953-3a87a2283a28 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:34.775679 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 561/1128] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:50:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 22:50:34.781520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb836050-ca94-49c2-ab7e-7fc83327e10b None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:34.874557 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb836050-ca94-49c2-ab7e-7fc83327e10b None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:34.906714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e754b27-9dad-4050-a095-89aab8c96420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:34.907546 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e754b27-9dad-4050-a095-89aab8c96420 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:34.907824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e754b27-9dad-4050-a095-89aab8c96420 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:34.908094 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e754b27-9dad-4050-a095-89aab8c96420 None admin] PATH_INFO: `/v3/projects/0ab5cc8d0bcd47f182e390bfe3867efa` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:34.909262 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e754b27-9dad-4050-a095-89aab8c96420 None admin] RBAC: Authorizing `identity:get_project(project_id=0ab5cc8d0bcd47f182e390bfe3867efa)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:34.910707 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e754b27-9dad-4050-a095-89aab8c96420 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:34.912358 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 568/1129] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:50:34 2022] GET /identity/v3/projects/0ab5cc8d0bcd47f182e390bfe3867efa => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:34.918418 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d9576-63e7-49da-b953-3a87a2283a28 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:35.010289 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d9576-63e7-49da-b953-3a87a2283a28 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:35.040730 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5307d-d80c-4375-a6a9-5b05796fca9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:35.041350 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5307d-d80c-4375-a6a9-5b05796fca9b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:35.041623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5307d-d80c-4375-a6a9-5b05796fca9b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:35.041889 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5307d-d80c-4375-a6a9-5b05796fca9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:35.042719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac5307d-d80c-4375-a6a9-5b05796fca9b None admin] RBAC: Adding query filter params (role.id=46ddc3cf4b124399a275bc639635e10a, scope.project.id=0ab5cc8d0bcd47f182e390bfe3867efa, user.id=1459bd8dfb294c5d9ffdc11426111737) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:35.043022 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac5307d-d80c-4375-a6a9-5b05796fca9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:35.044411 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac5307d-d80c-4375-a6a9-5b05796fca9b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:35.050366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 562/1130] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:50:34 2022] GET /identity/v3/role_assignments?user.id=1459bd8dfb294c5d9ffdc11426111737&scope.project.id=0ab5cc8d0bcd47f182e390bfe3867efa&role.id=46ddc3cf4b124399a275bc639635e10a => generated 608 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:37.273597 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b3890-ba49-4cdd-b777-d07668828eac None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:37.274175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b3890-ba49-4cdd-b777-d07668828eac None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:37.274175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b3890-ba49-4cdd-b777-d07668828eac None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:37.275524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 569/1131] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:50:37.280759 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61649d62-9652-4f90-87bf-12e8a9d527c3 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:37.281131 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61649d62-9652-4f90-87bf-12e8a9d527c3 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:37.281315 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61649d62-9652-4f90-87bf-12e8a9d527c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:37.297365 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-61649d62-9652-4f90-87bf-12e8a9d527c3 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:37.298556 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-61649d62-9652-4f90-87bf-12e8a9d527c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:37.360536 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 563/1132] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:50:37.383672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b3890-ba49-4cdd-b777-d07668828eac None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:37.478008 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b3890-ba49-4cdd-b777-d07668828eac None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:37.509659 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ff894-c6d6-487d-a216-9bf9acd96a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:37.510455 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ff894-c6d6-487d-a216-9bf9acd96a36 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:37.510733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ff894-c6d6-487d-a216-9bf9acd96a36 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:37.511003 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ff894-c6d6-487d-a216-9bf9acd96a36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:37.512145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ff894-c6d6-487d-a216-9bf9acd96a36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:37.513560 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ff894-c6d6-487d-a216-9bf9acd96a36 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:37.515175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 570/1133] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:50:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:37.520887 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61649d62-9652-4f90-87bf-12e8a9d527c3 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:37.613834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61649d62-9652-4f90-87bf-12e8a9d527c3 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:37.646072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8a4a2-2231-475f-9fb4-e089b8b5fc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:37.646764 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8a4a2-2231-475f-9fb4-e089b8b5fc6d None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:37.647044 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8a4a2-2231-475f-9fb4-e089b8b5fc6d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:37.647319 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8a4a2-2231-475f-9fb4-e089b8b5fc6d None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:37.648028 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8a4a2-2231-475f-9fb4-e089b8b5fc6d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:37.649481 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8a4a2-2231-475f-9fb4-e089b8b5fc6d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:37.673776 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 564/1134] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 9 22:50:37 2022] POST /identity/v3/projects => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 22:50:39.909212 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115ef3f-f96a-4f80-866a-035b8bc9430a None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:39.909733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115ef3f-f96a-4f80-866a-035b8bc9430a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:39.909733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115ef3f-f96a-4f80-866a-035b8bc9430a None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:39.911082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 571/1135] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:50:39.916306 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d83a71-f10d-4a9b-bcfe-68a9ea941553 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:39.916693 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d83a71-f10d-4a9b-bcfe-68a9ea941553 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:39.916866 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d83a71-f10d-4a9b-bcfe-68a9ea941553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:39.933542 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-29d83a71-f10d-4a9b-bcfe-68a9ea941553 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:39.934727 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-29d83a71-f10d-4a9b-bcfe-68a9ea941553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:39.995602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 565/1136] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:50:40.018835 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115ef3f-f96a-4f80-866a-035b8bc9430a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:40.114540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115ef3f-f96a-4f80-866a-035b8bc9430a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:40.145158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266515a-86ee-4360-8e34-0ad8e9e5d4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:40.145925 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266515a-86ee-4360-8e34-0ad8e9e5d4df None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:40.146242 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266515a-86ee-4360-8e34-0ad8e9e5d4df None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:40.146521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266515a-86ee-4360-8e34-0ad8e9e5d4df None admin] PATH_INFO: `/v3/domains/default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:40.147674 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266515a-86ee-4360-8e34-0ad8e9e5d4df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:40.149082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266515a-86ee-4360-8e34-0ad8e9e5d4df None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:40.150679 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 572/1137] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:50:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:40.156167 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d83a71-f10d-4a9b-bcfe-68a9ea941553 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:40.252199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d83a71-f10d-4a9b-bcfe-68a9ea941553 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:40.284362 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e37e8-935c-4b44-90a9-3cc6faea85b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:40.285012 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e37e8-935c-4b44-90a9-3cc6faea85b6 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:40.285337 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e37e8-935c-4b44-90a9-3cc6faea85b6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:40.285613 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e37e8-935c-4b44-90a9-3cc6faea85b6 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:40.286341 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e37e8-935c-4b44-90a9-3cc6faea85b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:40.287756 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e37e8-935c-4b44-90a9-3cc6faea85b6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:40.306758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 566/1138] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 9 22:50:40 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:50:42.571788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f942339-578a-48c3-a212-e3ae4578b0d5 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:42.572355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f942339-578a-48c3-a212-e3ae4578b0d5 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:42.572355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f942339-578a-48c3-a212-e3ae4578b0d5 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:42.573730 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 573/1139] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:50:42.578894 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc87ef-7cee-4594-a39f-eb22177866ec None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:42.579283 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc87ef-7cee-4594-a39f-eb22177866ec None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:42.579443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc87ef-7cee-4594-a39f-eb22177866ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:42.596586 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-28fc87ef-7cee-4594-a39f-eb22177866ec None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:42.597765 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-28fc87ef-7cee-4594-a39f-eb22177866ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:42.661381 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 567/1140] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:50:42.684351 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f942339-578a-48c3-a212-e3ae4578b0d5 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:42.778136 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f942339-578a-48c3-a212-e3ae4578b0d5 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:42.808709 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac8843-a87d-4560-b3be-7b49a9acb565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:42.809471 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac8843-a87d-4560-b3be-7b49a9acb565 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:42.809746 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac8843-a87d-4560-b3be-7b49a9acb565 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:42.810013 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac8843-a87d-4560-b3be-7b49a9acb565 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:42.821034 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfac8843-a87d-4560-b3be-7b49a9acb565 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:50:42.828524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-bfac8843-a87d-4560-b3be-7b49a9acb565 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:50:42.828524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:42.828524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:42.828524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:42.828524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:42.828524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:42.828524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:42.828524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:42.828524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:42.828524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:42.828524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:42.828524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:42.828524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:50:42.828524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:50:42.828524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:50:42.828524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:50:42.829836 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:50:42.829836 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:50:42.829836 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:50:42.829836 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:50:42.829836 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:42.829836 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:42.829836 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:42.829836 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:42.829836 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:42.829836 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:42.829836 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:42.829836 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:50:42.829836 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:42.829836 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:42.829836 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:42.829836 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:42.831624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:42.831624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:42.831624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:42.831624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:42.831624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:50:42.831624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:50:42.831624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:50:42.831624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:50:42.831624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:50:42.831624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:50:42.831624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:50:42.831624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:42.831624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 574/1141] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:50:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:50:42.836023 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc87ef-7cee-4594-a39f-eb22177866ec None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:42.926951 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc87ef-7cee-4594-a39f-eb22177866ec None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:42.958532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb979e0f-8125-4c81-a806-4c640cd1f77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:42.959172 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb979e0f-8125-4c81-a806-4c640cd1f77c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:42.959444 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb979e0f-8125-4c81-a806-4c640cd1f77c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:42.959709 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb979e0f-8125-4c81-a806-4c640cd1f77c None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:42.960432 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb979e0f-8125-4c81-a806-4c640cd1f77c None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:42.960727 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb979e0f-8125-4c81-a806-4c640cd1f77c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:42.962060 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb979e0f-8125-4c81-a806-4c640cd1f77c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:42.969574 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 568/1142] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:50:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:42.975220 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac8843-a87d-4560-b3be-7b49a9acb565 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:43.067523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac8843-a87d-4560-b3be-7b49a9acb565 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:43.098684 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a339a-4db8-4603-9b76-08e8ce80de84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:43.099461 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a339a-4db8-4603-9b76-08e8ce80de84 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:43.099737 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a339a-4db8-4603-9b76-08e8ce80de84 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:43.100008 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a339a-4db8-4603-9b76-08e8ce80de84 None admin] PATH_INFO: `/v3/users/9aff1658f58b4f09b9bc58b97292cfef` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:43.112927 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a339a-4db8-4603-9b76-08e8ce80de84 None admin] RBAC: Authorizing `identity:get_user(user_id=9aff1658f58b4f09b9bc58b97292cfef)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:43.114387 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a339a-4db8-4603-9b76-08e8ce80de84 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:43.115960 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 575/1143] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:50:42 2022] GET /identity/v3/users/9aff1658f58b4f09b9bc58b97292cfef => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:43.121345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb979e0f-8125-4c81-a806-4c640cd1f77c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:43.213213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb979e0f-8125-4c81-a806-4c640cd1f77c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:43.244401 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc96a26-fb0b-4abc-8ee3-51f562cbb666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:43.245022 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc96a26-fb0b-4abc-8ee3-51f562cbb666 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:43.245300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc96a26-fb0b-4abc-8ee3-51f562cbb666 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:43.245570 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc96a26-fb0b-4abc-8ee3-51f562cbb666 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:43.306927 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc96a26-fb0b-4abc-8ee3-51f562cbb666 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:43.308335 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc96a26-fb0b-4abc-8ee3-51f562cbb666 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:43.399494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 569/1144] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:50:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 22:50:43.405195 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a339a-4db8-4603-9b76-08e8ce80de84 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:43.497832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a339a-4db8-4603-9b76-08e8ce80de84 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:43.529307 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf00e1-8292-4b15-9535-ee2a4a33551c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:43.530158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf00e1-8292-4b15-9535-ee2a4a33551c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:43.530444 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf00e1-8292-4b15-9535-ee2a4a33551c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:43.530715 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf00e1-8292-4b15-9535-ee2a4a33551c None admin] PATH_INFO: `/v3/projects/735e9e12cf494d5bb1065d728518f027` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:43.531855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaf00e1-8292-4b15-9535-ee2a4a33551c None admin] RBAC: Authorizing `identity:get_project(project_id=735e9e12cf494d5bb1065d728518f027)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:43.533260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaf00e1-8292-4b15-9535-ee2a4a33551c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:43.534947 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 576/1145] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:50:43 2022] GET /identity/v3/projects/735e9e12cf494d5bb1065d728518f027 => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:43.540709 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc96a26-fb0b-4abc-8ee3-51f562cbb666 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:43.632966 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc96a26-fb0b-4abc-8ee3-51f562cbb666 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:43.665918 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83891b0f-0b48-4ba9-b032-ca8b18551369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:43.666534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83891b0f-0b48-4ba9-b032-ca8b18551369 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:43.666819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83891b0f-0b48-4ba9-b032-ca8b18551369 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:43.667089 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83891b0f-0b48-4ba9-b032-ca8b18551369 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:43.667908 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83891b0f-0b48-4ba9-b032-ca8b18551369 None admin] RBAC: Adding query filter params (role.id=bda550492f4f4c5c83f52fa89fc18a7d, scope.project.id=735e9e12cf494d5bb1065d728518f027, user.id=9aff1658f58b4f09b9bc58b97292cfef) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:43.668227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83891b0f-0b48-4ba9-b032-ca8b18551369 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:43.669638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83891b0f-0b48-4ba9-b032-ca8b18551369 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:43.675630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 570/1146] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:50:43 2022] GET /identity/v3/role_assignments?user.id=9aff1658f58b4f09b9bc58b97292cfef&scope.project.id=735e9e12cf494d5bb1065d728518f027&role.id=bda550492f4f4c5c83f52fa89fc18a7d => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:45.840368 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876585f0-2f31-4acf-b6ce-1dfc5cb0a8fd None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:45.840908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876585f0-2f31-4acf-b6ce-1dfc5cb0a8fd None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:45.840908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876585f0-2f31-4acf-b6ce-1dfc5cb0a8fd None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:45.842218 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 577/1147] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:50:45.847939 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d681e-56cc-4382-bfc3-9b23940e4e46 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:45.848314 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d681e-56cc-4382-bfc3-9b23940e4e46 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:45.848472 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d681e-56cc-4382-bfc3-9b23940e4e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:45.865232 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-4a0d681e-56cc-4382-bfc3-9b23940e4e46 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:45.866411 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-4a0d681e-56cc-4382-bfc3-9b23940e4e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:45.927610 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 571/1148] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:50:45.948949 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876585f0-2f31-4acf-b6ce-1dfc5cb0a8fd None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:46.044133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876585f0-2f31-4acf-b6ce-1dfc5cb0a8fd None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:46.075559 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd62ac3-821e-4239-87f5-16af1724b86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:46.076317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd62ac3-821e-4239-87f5-16af1724b86c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:46.076593 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd62ac3-821e-4239-87f5-16af1724b86c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:46.076859 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd62ac3-821e-4239-87f5-16af1724b86c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:46.086288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd62ac3-821e-4239-87f5-16af1724b86c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:50:46.093710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-4cd62ac3-821e-4239-87f5-16af1724b86c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:50:46.093710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:46.093710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:46.093710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:46.093710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:46.093710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:46.093710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:46.093710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:46.093710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:46.093710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:46.093710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:46.093710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:46.093710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:50:46.093710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:50:46.093710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:50:46.093710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:50:46.094853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:50:46.094853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:50:46.094853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:50:46.094853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:50:46.094853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:46.094853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:46.094853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:46.094853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:46.094853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:46.094853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:46.094853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:46.094853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:50:46.094853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:46.094853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:46.094853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:46.094853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:46.096558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:46.096558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:46.096558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:46.096558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:46.096558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:50:46.096558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:50:46.096558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:50:46.096558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:50:46.096558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:50:46.096558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:50:46.096558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:50:46.096558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:46.096558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 578/1149] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:50:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:50:46.101145 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d681e-56cc-4382-bfc3-9b23940e4e46 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:46.192309 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d681e-56cc-4382-bfc3-9b23940e4e46 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:46.224503 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd1c2e-eb22-47d7-b117-5b048d0da8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:46.225140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd1c2e-eb22-47d7-b117-5b048d0da8d7 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:46.225412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd1c2e-eb22-47d7-b117-5b048d0da8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:46.225677 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd1c2e-eb22-47d7-b117-5b048d0da8d7 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:46.226428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd1c2e-eb22-47d7-b117-5b048d0da8d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:46.226724 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd1c2e-eb22-47d7-b117-5b048d0da8d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:46.228060 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd1c2e-eb22-47d7-b117-5b048d0da8d7 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:46.236078 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 572/1150] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:50:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:46.241765 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd62ac3-821e-4239-87f5-16af1724b86c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:46.338980 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd62ac3-821e-4239-87f5-16af1724b86c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:46.371319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb09c73-9436-472f-ac0a-5b8888a2a461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:46.372165 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb09c73-9436-472f-ac0a-5b8888a2a461 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:46.372442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb09c73-9436-472f-ac0a-5b8888a2a461 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:46.372714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb09c73-9436-472f-ac0a-5b8888a2a461 None admin] PATH_INFO: `/v3/users/9aff1658f58b4f09b9bc58b97292cfef` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:46.373880 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb09c73-9436-472f-ac0a-5b8888a2a461 None admin] RBAC: Authorizing `identity:get_user(user_id=9aff1658f58b4f09b9bc58b97292cfef)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:46.375331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb09c73-9436-472f-ac0a-5b8888a2a461 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:46.376918 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 579/1151] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:50:46 2022] GET /identity/v3/users/9aff1658f58b4f09b9bc58b97292cfef => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:46.382997 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd1c2e-eb22-47d7-b117-5b048d0da8d7 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:46.474371 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd1c2e-eb22-47d7-b117-5b048d0da8d7 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:46.505881 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba7fba-f160-4802-aa12-bb8b2d505d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:46.506732 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba7fba-f160-4802-aa12-bb8b2d505d2b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:46.507008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba7fba-f160-4802-aa12-bb8b2d505d2b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:46.507276 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba7fba-f160-4802-aa12-bb8b2d505d2b None admin] PATH_INFO: `/v3/projects/735e9e12cf494d5bb1065d728518f027` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:46.508414 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba7fba-f160-4802-aa12-bb8b2d505d2b None admin] RBAC: Authorizing `identity:get_project(project_id=735e9e12cf494d5bb1065d728518f027)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:46.509812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba7fba-f160-4802-aa12-bb8b2d505d2b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:46.511461 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 573/1152] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:50:46 2022] GET /identity/v3/projects/735e9e12cf494d5bb1065d728518f027 => generated 311 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:46.517063 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb09c73-9436-472f-ac0a-5b8888a2a461 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:46.614481 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb09c73-9436-472f-ac0a-5b8888a2a461 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:46.645790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bcbc0f-320e-4e2b-aec4-935857872048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:46.646700 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcbc0f-320e-4e2b-aec4-935857872048 None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:46.646977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcbc0f-320e-4e2b-aec4-935857872048 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:46.647250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcbc0f-320e-4e2b-aec4-935857872048 None admin] PATH_INFO: `/v3/projects/735e9e12cf494d5bb1065d728518f027/users/9aff1658f58b4f09b9bc58b97292cfef/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:46.649178 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bcbc0f-320e-4e2b-aec4-935857872048 None admin] RBAC: Authorizing `identity:create_grant(project_id=735e9e12cf494d5bb1065d728518f027, user_id=9aff1658f58b4f09b9bc58b97292cfef, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:46.650638 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bcbc0f-320e-4e2b-aec4-935857872048 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:46.664171 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 580/1153] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:50:46 2022] PUT /identity/v3/projects/735e9e12cf494d5bb1065d728518f027/users/9aff1658f58b4f09b9bc58b97292cfef/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:50:48.829374 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269c9b0-4e07-4032-8b39-741539859572 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:48.829915 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269c9b0-4e07-4032-8b39-741539859572 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:48.829915 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269c9b0-4e07-4032-8b39-741539859572 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:48.831963 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 574/1154] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:50:48.837136 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209a968-4548-4709-a10d-497dfb33a034 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:48.837551 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209a968-4548-4709-a10d-497dfb33a034 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:48.837660 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209a968-4548-4709-a10d-497dfb33a034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:48.854371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-4209a968-4548-4709-a10d-497dfb33a034 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:48.855528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4209a968-4548-4709-a10d-497dfb33a034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:48.918649 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 581/1155] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:50:48.942532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269c9b0-4e07-4032-8b39-741539859572 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:49.039961 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269c9b0-4e07-4032-8b39-741539859572 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:49.071142 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2cf31-b924-4dfa-adc8-1d282d99c101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:49.071914 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2cf31-b924-4dfa-adc8-1d282d99c101 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:49.072193 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2cf31-b924-4dfa-adc8-1d282d99c101 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:49.072468 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2cf31-b924-4dfa-adc8-1d282d99c101 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:49.082956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a2cf31-b924-4dfa-adc8-1d282d99c101 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:50:49.090466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-86a2cf31-b924-4dfa-adc8-1d282d99c101 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:50:49.090466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:49.090466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:49.090466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:49.090466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:49.090466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:49.090466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:49.090466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:49.090466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:49.090466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:49.090466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:49.090466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:49.090466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:50:49.090466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:50:49.090466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:50:49.090466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:50:49.091566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:50:49.091566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:50:49.091566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:50:49.091566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:50:49.091566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:49.091566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:49.091566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:49.091566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:49.091566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:49.091566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:49.091566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:49.091566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:50:49.091566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:49.091566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:49.091566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:49.091566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:49.092658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:49.092658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:49.092658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:49.092658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:49.092658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:50:49.092658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:50:49.092658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:50:49.092658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:50:49.092658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:50:49.092658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:50:49.092658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:50:49.092658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:50:49.092658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 575/1156] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:50:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:50:49.097736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209a968-4548-4709-a10d-497dfb33a034 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:49.190473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209a968-4548-4709-a10d-497dfb33a034 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:49.220874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c1bb6-0843-4466-bec7-db2548602474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:49.221519 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c1bb6-0843-4466-bec7-db2548602474 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:49.221790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c1bb6-0843-4466-bec7-db2548602474 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:49.222053 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c1bb6-0843-4466-bec7-db2548602474 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:49.222803 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c1bb6-0843-4466-bec7-db2548602474 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:49.223100 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c1bb6-0843-4466-bec7-db2548602474 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:49.224441 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c1bb6-0843-4466-bec7-db2548602474 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:49.231963 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 582/1157] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:50:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:50:49.238381 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2cf31-b924-4dfa-adc8-1d282d99c101 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:49.332469 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2cf31-b924-4dfa-adc8-1d282d99c101 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:49.364106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084c4c8-5d5f-44e9-a7c9-1a364926a771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:49.364948 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084c4c8-5d5f-44e9-a7c9-1a364926a771 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:49.365221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084c4c8-5d5f-44e9-a7c9-1a364926a771 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:49.365486 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084c4c8-5d5f-44e9-a7c9-1a364926a771 None admin] PATH_INFO: `/v3/users/9aff1658f58b4f09b9bc58b97292cfef` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:49.366673 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2084c4c8-5d5f-44e9-a7c9-1a364926a771 None admin] RBAC: Authorizing `identity:get_user(user_id=9aff1658f58b4f09b9bc58b97292cfef)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:49.368089 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2084c4c8-5d5f-44e9-a7c9-1a364926a771 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:49.369673 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 576/1158] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:50:49 2022] GET /identity/v3/users/9aff1658f58b4f09b9bc58b97292cfef => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:49.375461 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c1bb6-0843-4466-bec7-db2548602474 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:49.468119 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c1bb6-0843-4466-bec7-db2548602474 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:49.499446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd85d13-b380-4cff-a917-20acc6022270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:49.500077 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd85d13-b380-4cff-a917-20acc6022270 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:49.500355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd85d13-b380-4cff-a917-20acc6022270 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:49.500628 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd85d13-b380-4cff-a917-20acc6022270 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:49.563220 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd85d13-b380-4cff-a917-20acc6022270 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:49.565246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd85d13-b380-4cff-a917-20acc6022270 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:49.656633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 583/1159] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:50:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 22:50:49.662981 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084c4c8-5d5f-44e9-a7c9-1a364926a771 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:49.754524 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084c4c8-5d5f-44e9-a7c9-1a364926a771 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:49.785884 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa55dc-349d-4c7a-9568-decd7ea4b42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:49.786740 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa55dc-349d-4c7a-9568-decd7ea4b42d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:49.787017 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa55dc-349d-4c7a-9568-decd7ea4b42d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:49.787286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa55dc-349d-4c7a-9568-decd7ea4b42d None admin] PATH_INFO: `/v3/projects/735e9e12cf494d5bb1065d728518f027` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:49.788434 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa55dc-349d-4c7a-9568-decd7ea4b42d None admin] RBAC: Authorizing `identity:get_project(project_id=735e9e12cf494d5bb1065d728518f027)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:49.789847 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa55dc-349d-4c7a-9568-decd7ea4b42d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:49.791518 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 577/1160] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:50:49 2022] GET /identity/v3/projects/735e9e12cf494d5bb1065d728518f027 => generated 311 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:49.797279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd85d13-b380-4cff-a917-20acc6022270 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:49.892409 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd85d13-b380-4cff-a917-20acc6022270 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:49.924517 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839dce2-33d3-49dd-8620-5530c0a5b0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:49.925153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839dce2-33d3-49dd-8620-5530c0a5b0a3 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:49.925428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839dce2-33d3-49dd-8620-5530c0a5b0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:49.925696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839dce2-33d3-49dd-8620-5530c0a5b0a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:49.926534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839dce2-33d3-49dd-8620-5530c0a5b0a3 None admin] RBAC: Adding query filter params (role.id=bda550492f4f4c5c83f52fa89fc18a7d, scope.project.id=735e9e12cf494d5bb1065d728518f027, user.id=9aff1658f58b4f09b9bc58b97292cfef) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:49.926842 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839dce2-33d3-49dd-8620-5530c0a5b0a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:49.928262 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839dce2-33d3-49dd-8620-5530c0a5b0a3 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:49.933778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 584/1161] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:50:49 2022] GET /identity/v3/role_assignments?user.id=9aff1658f58b4f09b9bc58b97292cfef&scope.project.id=735e9e12cf494d5bb1065d728518f027&role.id=bda550492f4f4c5c83f52fa89fc18a7d => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:52.177377 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df1bf9-514a-4752-b3de-42c223a45f03 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:52.177936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df1bf9-514a-4752-b3de-42c223a45f03 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:52.177936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df1bf9-514a-4752-b3de-42c223a45f03 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:52.179283 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 578/1162] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:50:52.184514 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191d7b5-7f0a-420d-884f-6db10c6f12c8 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:52.184957 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191d7b5-7f0a-420d-884f-6db10c6f12c8 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:52.185074 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191d7b5-7f0a-420d-884f-6db10c6f12c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:52.201192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-9191d7b5-7f0a-420d-884f-6db10c6f12c8 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:52.202382 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9191d7b5-7f0a-420d-884f-6db10c6f12c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:52.265097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 585/1163] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:50:52.287675 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df1bf9-514a-4752-b3de-42c223a45f03 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:52.380633 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df1bf9-514a-4752-b3de-42c223a45f03 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:52.412381 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe7a32-a4ab-4b17-a7b3-a62852a6186e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:52.412995 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe7a32-a4ab-4b17-a7b3-a62852a6186e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:52.413270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe7a32-a4ab-4b17-a7b3-a62852a6186e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:52.413537 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe7a32-a4ab-4b17-a7b3-a62852a6186e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:52.475288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe7a32-a4ab-4b17-a7b3-a62852a6186e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:52.476649 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe7a32-a4ab-4b17-a7b3-a62852a6186e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:52.568266 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 579/1164] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:50:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 22:50:52.574320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191d7b5-7f0a-420d-884f-6db10c6f12c8 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:52.666991 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191d7b5-7f0a-420d-884f-6db10c6f12c8 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:52.698465 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695717b-9829-41a3-b482-b2a02ac25a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:52.699243 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695717b-9829-41a3-b482-b2a02ac25a25 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:52.699523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695717b-9829-41a3-b482-b2a02ac25a25 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:52.699795 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695717b-9829-41a3-b482-b2a02ac25a25 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:52.706196 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695717b-9829-41a3-b482-b2a02ac25a25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:52.707642 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695717b-9829-41a3-b482-b2a02ac25a25 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:52.721733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-1695717b-9829-41a3-b482-b2a02ac25a25 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 09 22:50:52.721733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:52.721733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 09 22:50:52.721733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 09 22:50:52.721733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:50:52.721733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:50:52.721733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:50:52.721733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:50:52.721733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 09 22:50:52.721733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:52.721733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:50:52.721733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:52.721733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:50:52.721733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:50:52.721733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:50:52.721733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:50:52.721733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:50:52.723024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:50:52.723024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:50:52.723024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:50:52.723024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:50:52.723024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:50:52.723024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:50:52.723024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:50:52.723024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:50:52.723024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:50:52.723024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:50:52.723024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:50:52.723024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:50:52.723024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:50:52.723024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:50:52.723024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:50:52.723024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:50:52.724052 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:50:52.724052 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:50:52.724052 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:50:52.724052 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:50:52.724052 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:50:52.724052 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:50:52.724052 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:50:52.724052 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:50:52.724052 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:50:52.724052 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:50:52.724052 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 09 22:50:52.724052 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:50:52.724052 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 09 22:50:52.724052 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:50:52.724052 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 586/1165] 10.222.0.23 () {56 vars in 1117 bytes} [Mon May 9 22:50:52 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:50:52.728802 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe7a32-a4ab-4b17-a7b3-a62852a6186e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:52.821535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe7a32-a4ab-4b17-a7b3-a62852a6186e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:52.853248 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57b1b4-3821-4b18-babf-f6a4843c0ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:52.853890 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57b1b4-3821-4b18-babf-f6a4843c0ee0 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:52.854197 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57b1b4-3821-4b18-babf-f6a4843c0ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:52.854474 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57b1b4-3821-4b18-babf-f6a4843c0ee0 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:52.855184 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57b1b4-3821-4b18-babf-f6a4843c0ee0 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:50:52.855492 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57b1b4-3821-4b18-babf-f6a4843c0ee0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:52.856845 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57b1b4-3821-4b18-babf-f6a4843c0ee0 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:52.864005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 580/1166] 10.222.0.23 () {56 vars in 1115 bytes} [Mon May 9 22:50:52 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:55.069027 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc64af2-7579-470f-adbc-fe6a7c9c7cdf None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:55.069547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc64af2-7579-470f-adbc-fe6a7c9c7cdf None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:55.069547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc64af2-7579-470f-adbc-fe6a7c9c7cdf None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:55.070943 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 587/1167] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:50:55.076174 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a74d8-35da-4e40-8d86-dbf38f53f582 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:55.076574 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a74d8-35da-4e40-8d86-dbf38f53f582 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:55.076711 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a74d8-35da-4e40-8d86-dbf38f53f582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:55.092613 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e66a74d8-35da-4e40-8d86-dbf38f53f582 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:55.093765 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e66a74d8-35da-4e40-8d86-dbf38f53f582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:55.154846 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 581/1168] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 22:50:55.177886 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc64af2-7579-470f-adbc-fe6a7c9c7cdf None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:55.272272 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc64af2-7579-470f-adbc-fe6a7c9c7cdf None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:55.302895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd90b4-b95a-44b6-89b2-0f3f4f600f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:55.303524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd90b4-b95a-44b6-89b2-0f3f4f600f2a None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:55.303799 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd90b4-b95a-44b6-89b2-0f3f4f600f2a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:55.304068 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd90b4-b95a-44b6-89b2-0f3f4f600f2a None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:55.304790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fd90b4-b95a-44b6-89b2-0f3f4f600f2a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:55.306157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fd90b4-b95a-44b6-89b2-0f3f4f600f2a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:55.319715 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 588/1169] 10.222.0.23 () {60 vars in 1138 bytes} [Mon May 9 22:50:55 2022] POST /identity/v3/domains => generated 267 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:50:57.553436 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8484b2f-7050-46a7-abf7-aadf01c4d6e9 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:57.554073 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8484b2f-7050-46a7-abf7-aadf01c4d6e9 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:57.554073 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8484b2f-7050-46a7-abf7-aadf01c4d6e9 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:57.555387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 582/1170] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:50:57.560516 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527cd5f-5230-4208-add1-41d5c16abb0a None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:57.560965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527cd5f-5230-4208-add1-41d5c16abb0a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:57.561066 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527cd5f-5230-4208-add1-41d5c16abb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:57.577801 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5527cd5f-5230-4208-add1-41d5c16abb0a None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:50:57.578996 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5527cd5f-5230-4208-add1-41d5c16abb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:50:57.642006 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 589/1171] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:50:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:50:57.664912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8484b2f-7050-46a7-abf7-aadf01c4d6e9 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:57.759934 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8484b2f-7050-46a7-abf7-aadf01c4d6e9 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:57.790616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767eea6c-9930-42e5-b54e-8c792e2430b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:57.791441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767eea6c-9930-42e5-b54e-8c792e2430b0 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:57.791717 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767eea6c-9930-42e5-b54e-8c792e2430b0 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:57.791985 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767eea6c-9930-42e5-b54e-8c792e2430b0 None admin] PATH_INFO: `/v3/domains/ba2e55e731bf489da30a4be1a0d274ca` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:57.802247 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767eea6c-9930-42e5-b54e-8c792e2430b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ba2e55e731bf489da30a4be1a0d274ca)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:57.803713 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767eea6c-9930-42e5-b54e-8c792e2430b0 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:57.805240 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 583/1172] 10.222.0.23 () {56 vars in 1183 bytes} [Mon May 9 22:50:57 2022] GET /identity/v3/domains/ba2e55e731bf489da30a4be1a0d274ca => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:50:57.810811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527cd5f-5230-4208-add1-41d5c16abb0a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:50:57.901688 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527cd5f-5230-4208-add1-41d5c16abb0a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:50:57.933551 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c785cc-edf2-4021-a83d-ec10bea63a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:50:57.934225 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c785cc-edf2-4021-a83d-ec10bea63a22 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:50:57.934501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c785cc-edf2-4021-a83d-ec10bea63a22 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:50:57.934767 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c785cc-edf2-4021-a83d-ec10bea63a22 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:50:57.935462 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c785cc-edf2-4021-a83d-ec10bea63a22 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:50:57.936871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c785cc-edf2-4021-a83d-ec10bea63a22 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:50:57.960186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 590/1173] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 9 22:50:57 2022] POST /identity/v3/projects => generated 361 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:51:00.157541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4069f2a-da0e-4f95-a898-72085009f7ac None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:00.158637 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4069f2a-da0e-4f95-a898-72085009f7ac None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:00.158637 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4069f2a-da0e-4f95-a898-72085009f7ac None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:00.159982 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 584/1174] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:51:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:51:00.165096 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7505faf-3cf7-45b9-9799-9963b8ecdc21 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:00.165522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7505faf-3cf7-45b9-9799-9963b8ecdc21 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:00.165646 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7505faf-3cf7-45b9-9799-9963b8ecdc21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:00.182233 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e7505faf-3cf7-45b9-9799-9963b8ecdc21 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:51:00.183380 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e7505faf-3cf7-45b9-9799-9963b8ecdc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:51:00.243908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 591/1175] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:51:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:51:00.267233 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4069f2a-da0e-4f95-a898-72085009f7ac None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:51:00.362364 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4069f2a-da0e-4f95-a898-72085009f7ac None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:51:00.392854 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447b8ac-9dba-4906-ac93-a5fbff4a6b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:51:00.393689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447b8ac-9dba-4906-ac93-a5fbff4a6b55 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:00.393967 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447b8ac-9dba-4906-ac93-a5fbff4a6b55 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:00.394270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447b8ac-9dba-4906-ac93-a5fbff4a6b55 None admin] PATH_INFO: `/v3/domains/ba2e55e731bf489da30a4be1a0d274ca` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:00.395421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447b8ac-9dba-4906-ac93-a5fbff4a6b55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ba2e55e731bf489da30a4be1a0d274ca)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:51:00.396835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447b8ac-9dba-4906-ac93-a5fbff4a6b55 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:51:00.398436 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 585/1176] 10.222.0.23 () {56 vars in 1183 bytes} [Mon May 9 22:51:00 2022] GET /identity/v3/domains/ba2e55e731bf489da30a4be1a0d274ca => generated 267 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:51:00.404015 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7505faf-3cf7-45b9-9799-9963b8ecdc21 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:51:00.495695 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7505faf-3cf7-45b9-9799-9963b8ecdc21 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:51:00.527694 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0711a3d-4544-443f-ae78-652dfcc51941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:51:00.528337 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0711a3d-4544-443f-ae78-652dfcc51941 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:00.528612 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0711a3d-4544-443f-ae78-652dfcc51941 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:00.528878 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0711a3d-4544-443f-ae78-652dfcc51941 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:00.529560 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0711a3d-4544-443f-ae78-652dfcc51941 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:51:00.530993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0711a3d-4544-443f-ae78-652dfcc51941 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:51:00.550133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 592/1177] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 9 22:51:00 2022] POST /identity/v3/users => generated 311 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:51:02.773575 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e28a06-ef98-42d4-a20c-09c09b0d4029 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:02.774140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e28a06-ef98-42d4-a20c-09c09b0d4029 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:02.774140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e28a06-ef98-42d4-a20c-09c09b0d4029 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:02.775497 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 586/1178] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:51:02.780595 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48ae87-58a1-45f0-8f50-f8b6a3504033 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:02.781034 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48ae87-58a1-45f0-8f50-f8b6a3504033 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:02.781136 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48ae87-58a1-45f0-8f50-f8b6a3504033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:02.797874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3a48ae87-58a1-45f0-8f50-f8b6a3504033 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:51:02.799596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3a48ae87-58a1-45f0-8f50-f8b6a3504033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:51:02.862281 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 593/1179] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:51:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:51:02.885201 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e28a06-ef98-42d4-a20c-09c09b0d4029 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:51:02.979077 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e28a06-ef98-42d4-a20c-09c09b0d4029 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:51:03.011106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9747c5-1a4f-4db0-a998-58b4b1292635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:51:03.011879 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9747c5-1a4f-4db0-a998-58b4b1292635 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:03.012157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9747c5-1a4f-4db0-a998-58b4b1292635 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:03.012425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9747c5-1a4f-4db0-a998-58b4b1292635 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:03.022517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9747c5-1a4f-4db0-a998-58b4b1292635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:51:03.030001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-7e9747c5-1a4f-4db0-a998-58b4b1292635 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:51:03.030001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:51:03.030001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:51:03.030001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:51:03.030001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:51:03.030001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:51:03.030001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:51:03.030001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:51:03.030001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:51:03.030001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:51:03.030001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:51:03.030001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:51:03.030001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:51:03.030001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:51:03.030001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:51:03.030001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:51:03.031195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:51:03.031195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:51:03.031195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:51:03.031195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:51:03.031195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:51:03.031195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:51:03.031195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:51:03.031195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:51:03.031195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:51:03.031195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:51:03.031195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:51:03.031195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:51:03.031195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:51:03.031195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:51:03.031195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:51:03.031195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:51:03.032301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:51:03.032301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:51:03.032301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:51:03.032301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:51:03.032301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:51:03.032301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:51:03.032301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:51:03.032301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:51:03.032301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:51:03.032301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:51:03.032301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:51:03.032301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:51:03.032301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 587/1180] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:51:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:51:03.037160 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48ae87-58a1-45f0-8f50-f8b6a3504033 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:51:03.129623 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48ae87-58a1-45f0-8f50-f8b6a3504033 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:51:03.161429 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a6a80-4bfd-4199-bf14-b04a3f1f64ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:51:03.162026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a6a80-4bfd-4199-bf14-b04a3f1f64ab None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:03.162373 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a6a80-4bfd-4199-bf14-b04a3f1f64ab None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:03.162651 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a6a80-4bfd-4199-bf14-b04a3f1f64ab None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:03.163385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a6a80-4bfd-4199-bf14-b04a3f1f64ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:51:03.163687 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a6a80-4bfd-4199-bf14-b04a3f1f64ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:51:03.165060 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a6a80-4bfd-4199-bf14-b04a3f1f64ab None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:51:03.172668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 594/1181] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:51:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:51:03.178785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9747c5-1a4f-4db0-a998-58b4b1292635 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:51:03.271526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9747c5-1a4f-4db0-a998-58b4b1292635 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:51:03.303587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b35580-1141-4fb3-a5e2-7eabe68c51d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:51:03.304428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b35580-1141-4fb3-a5e2-7eabe68c51d6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:03.304706 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b35580-1141-4fb3-a5e2-7eabe68c51d6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:03.304976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b35580-1141-4fb3-a5e2-7eabe68c51d6 None admin] PATH_INFO: `/v3/users/455fce54edf64bb6bee0ad83a733a052` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:03.317134 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b35580-1141-4fb3-a5e2-7eabe68c51d6 None admin] RBAC: Authorizing `identity:get_user(user_id=455fce54edf64bb6bee0ad83a733a052)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:51:03.318585 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b35580-1141-4fb3-a5e2-7eabe68c51d6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:51:03.320190 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 588/1182] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:51:03 2022] GET /identity/v3/users/455fce54edf64bb6bee0ad83a733a052 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:51:03.325821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a6a80-4bfd-4199-bf14-b04a3f1f64ab None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:51:03.419669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a6a80-4bfd-4199-bf14-b04a3f1f64ab None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:51:03.450771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3d637-757f-4a69-917c-7e18521b8e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:51:03.451400 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3d637-757f-4a69-917c-7e18521b8e3d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:03.451679 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3d637-757f-4a69-917c-7e18521b8e3d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:03.451950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3d637-757f-4a69-917c-7e18521b8e3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:03.514720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3d637-757f-4a69-917c-7e18521b8e3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:51:03.516147 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3d637-757f-4a69-917c-7e18521b8e3d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:51:03.607194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 595/1183] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:51:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 22:51:03.613165 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b35580-1141-4fb3-a5e2-7eabe68c51d6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:51:03.705866 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b35580-1141-4fb3-a5e2-7eabe68c51d6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:51:03.737390 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0709d3-a1ae-47f6-9fce-c4f27dd93197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:51:03.738248 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0709d3-a1ae-47f6-9fce-c4f27dd93197 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:03.738524 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0709d3-a1ae-47f6-9fce-c4f27dd93197 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:03.738795 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0709d3-a1ae-47f6-9fce-c4f27dd93197 None admin] PATH_INFO: `/v3/projects/978da135c350476ba9cb078940ff0feb` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:03.739925 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0709d3-a1ae-47f6-9fce-c4f27dd93197 None admin] RBAC: Authorizing `identity:get_project(project_id=978da135c350476ba9cb078940ff0feb)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:51:03.741331 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0709d3-a1ae-47f6-9fce-c4f27dd93197 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:51:03.742995 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 589/1184] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:51:03 2022] GET /identity/v3/projects/978da135c350476ba9cb078940ff0feb => generated 361 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:51:03.749040 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3d637-757f-4a69-917c-7e18521b8e3d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:51:03.840241 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3d637-757f-4a69-917c-7e18521b8e3d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:51:03.872811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40854c5c-3e50-428d-8045-8677e4567eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:51:03.873452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40854c5c-3e50-428d-8045-8677e4567eda None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:03.873731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40854c5c-3e50-428d-8045-8677e4567eda None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:03.874002 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40854c5c-3e50-428d-8045-8677e4567eda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:03.874848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40854c5c-3e50-428d-8045-8677e4567eda None admin] RBAC: Adding query filter params (role.id=bda550492f4f4c5c83f52fa89fc18a7d, scope.project.id=978da135c350476ba9cb078940ff0feb, user.id=455fce54edf64bb6bee0ad83a733a052) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:51:03.875155 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40854c5c-3e50-428d-8045-8677e4567eda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:51:03.876570 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40854c5c-3e50-428d-8045-8677e4567eda None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:51:03.881815 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 596/1185] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:51:03 2022] GET /identity/v3/role_assignments?user.id=455fce54edf64bb6bee0ad83a733a052&scope.project.id=978da135c350476ba9cb078940ff0feb&role.id=bda550492f4f4c5c83f52fa89fc18a7d => generated 259 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:51:06.063575 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7aed1-a649-4164-93e8-f43f4cb315c0 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:06.064662 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7aed1-a649-4164-93e8-f43f4cb315c0 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:06.064767 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7aed1-a649-4164-93e8-f43f4cb315c0 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:06.066047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 590/1186] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:51:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:51:06.071133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb37df-4099-415d-a54c-f0e5307be80f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:06.071539 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb37df-4099-415d-a54c-f0e5307be80f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:06.071687 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb37df-4099-415d-a54c-f0e5307be80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:06.088450 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-9cfb37df-4099-415d-a54c-f0e5307be80f None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:51:06.089614 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9cfb37df-4099-415d-a54c-f0e5307be80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:51:06.151324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 597/1187] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:51:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:51:06.174324 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7aed1-a649-4164-93e8-f43f4cb315c0 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:51:06.268440 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7aed1-a649-4164-93e8-f43f4cb315c0 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:51:06.309818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e27cb7-1e9a-423f-881f-2e45ab903e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:51:06.310627 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e27cb7-1e9a-423f-881f-2e45ab903e8a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:06.310907 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e27cb7-1e9a-423f-881f-2e45ab903e8a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:06.311180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e27cb7-1e9a-423f-881f-2e45ab903e8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:06.320774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e27cb7-1e9a-423f-881f-2e45ab903e8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:51:06.328253 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-b0e27cb7-1e9a-423f-881f-2e45ab903e8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:51:06.328253 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:51:06.328253 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:51:06.328253 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:51:06.328253 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:51:06.328253 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:51:06.328253 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:51:06.328253 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:51:06.328253 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:51:06.328253 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:51:06.328253 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:51:06.328253 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:51:06.328253 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:51:06.328253 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:51:06.328253 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:51:06.328253 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:51:06.329363 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:51:06.329363 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:51:06.329363 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:51:06.329363 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:51:06.329363 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:51:06.329363 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:51:06.329363 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:51:06.329363 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:51:06.329363 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:51:06.329363 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:51:06.329363 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:51:06.329363 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:51:06.329363 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:51:06.329363 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:51:06.329363 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:51:06.329363 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:51:06.330428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:51:06.330428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:51:06.330428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:51:06.330428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:51:06.330428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:51:06.330428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:51:06.330428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:51:06.330428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:51:06.330428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:51:06.330428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:51:06.330428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:51:06.330428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:51:06.330428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 591/1188] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:51:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:51:06.335492 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb37df-4099-415d-a54c-f0e5307be80f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:51:06.428090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb37df-4099-415d-a54c-f0e5307be80f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:51:06.459157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ae81a-bd39-4165-8467-bef0118dfaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:51:06.459805 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ae81a-bd39-4165-8467-bef0118dfaa1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:06.460086 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ae81a-bd39-4165-8467-bef0118dfaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:06.460355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ae81a-bd39-4165-8467-bef0118dfaa1 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:06.461081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ae81a-bd39-4165-8467-bef0118dfaa1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:51:06.461378 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ae81a-bd39-4165-8467-bef0118dfaa1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:51:06.462787 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ae81a-bd39-4165-8467-bef0118dfaa1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:51:06.470936 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 598/1189] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:51:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:51:06.476796 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e27cb7-1e9a-423f-881f-2e45ab903e8a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:51:06.569367 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e27cb7-1e9a-423f-881f-2e45ab903e8a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:51:06.601360 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e4b9e-6088-4c1c-b767-d8f495a40ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:51:06.602169 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e4b9e-6088-4c1c-b767-d8f495a40ba4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:06.602452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e4b9e-6088-4c1c-b767-d8f495a40ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:06.602724 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e4b9e-6088-4c1c-b767-d8f495a40ba4 None admin] PATH_INFO: `/v3/users/455fce54edf64bb6bee0ad83a733a052` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:06.603895 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e4b9e-6088-4c1c-b767-d8f495a40ba4 None admin] RBAC: Authorizing `identity:get_user(user_id=455fce54edf64bb6bee0ad83a733a052)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:51:06.605307 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e4b9e-6088-4c1c-b767-d8f495a40ba4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:51:06.606934 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 592/1190] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:51:06 2022] GET /identity/v3/users/455fce54edf64bb6bee0ad83a733a052 => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:51:06.612754 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ae81a-bd39-4165-8467-bef0118dfaa1 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:51:06.708221 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ae81a-bd39-4165-8467-bef0118dfaa1 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:51:06.739354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1502ef1-628c-47a7-bb2c-86b0fd09bd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:51:06.740187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1502ef1-628c-47a7-bb2c-86b0fd09bd98 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:06.740461 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1502ef1-628c-47a7-bb2c-86b0fd09bd98 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:06.740730 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1502ef1-628c-47a7-bb2c-86b0fd09bd98 None admin] PATH_INFO: `/v3/projects/978da135c350476ba9cb078940ff0feb` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:06.741867 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1502ef1-628c-47a7-bb2c-86b0fd09bd98 None admin] RBAC: Authorizing `identity:get_project(project_id=978da135c350476ba9cb078940ff0feb)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:51:06.743306 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1502ef1-628c-47a7-bb2c-86b0fd09bd98 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:51:06.744967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 599/1191] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:51:06 2022] GET /identity/v3/projects/978da135c350476ba9cb078940ff0feb => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:51:06.750812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e4b9e-6088-4c1c-b767-d8f495a40ba4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:51:06.844441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e4b9e-6088-4c1c-b767-d8f495a40ba4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:51:06.875310 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e26316-0931-4b07-b191-a094fdf57f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:51:06.876110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e26316-0931-4b07-b191-a094fdf57f89 None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:06.876952 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e26316-0931-4b07-b191-a094fdf57f89 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:06.877224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e26316-0931-4b07-b191-a094fdf57f89 None admin] PATH_INFO: `/v3/projects/978da135c350476ba9cb078940ff0feb/users/455fce54edf64bb6bee0ad83a733a052/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:06.879182 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e26316-0931-4b07-b191-a094fdf57f89 None admin] RBAC: Authorizing `identity:create_grant(project_id=978da135c350476ba9cb078940ff0feb, user_id=455fce54edf64bb6bee0ad83a733a052, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:51:06.880592 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e26316-0931-4b07-b191-a094fdf57f89 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:51:06.893038 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 593/1192] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:51:06 2022] PUT /identity/v3/projects/978da135c350476ba9cb078940ff0feb/users/455fce54edf64bb6bee0ad83a733a052/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:51:09.074131 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2849ef8c-e462-4210-bd65-c10635b887ac None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:09.074650 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2849ef8c-e462-4210-bd65-c10635b887ac None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:09.074763 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2849ef8c-e462-4210-bd65-c10635b887ac None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:09.076106 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 600/1193] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:51:09.081755 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74287c-be19-4080-b481-f7b6dfc0a374 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:09.082213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74287c-be19-4080-b481-f7b6dfc0a374 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:09.082332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74287c-be19-4080-b481-f7b6dfc0a374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:09.099027 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-9a74287c-be19-4080-b481-f7b6dfc0a374 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:51:09.100202 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9a74287c-be19-4080-b481-f7b6dfc0a374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:51:09.163033 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 594/1194] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:51:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:51:09.186665 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2849ef8c-e462-4210-bd65-c10635b887ac None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:51:09.281110 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2849ef8c-e462-4210-bd65-c10635b887ac None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:51:09.312628 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b03c79-ac9b-4436-9546-b82acefff439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:51:09.313399 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b03c79-ac9b-4436-9546-b82acefff439 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:09.313677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b03c79-ac9b-4436-9546-b82acefff439 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:09.313973 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b03c79-ac9b-4436-9546-b82acefff439 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:09.323868 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b03c79-ac9b-4436-9546-b82acefff439 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:51:09.331459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-42b03c79-ac9b-4436-9546-b82acefff439 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:51:09.331459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:51:09.331459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:51:09.331459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:51:09.331459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:51:09.331459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:51:09.331459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:51:09.331459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:51:09.331459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:51:09.331459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:51:09.331459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:51:09.331459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:51:09.331459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:51:09.331459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:51:09.331459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:51:09.331459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:51:09.332543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:51:09.332543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:51:09.332543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:51:09.332543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:51:09.332543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:51:09.332543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:51:09.332543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:51:09.332543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:51:09.332543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:51:09.332543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:51:09.332543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:51:09.332543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:51:09.332543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:51:09.332543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:51:09.332543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:51:09.332543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:51:09.333426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:51:09.333426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:51:09.333426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:51:09.333426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:51:09.333426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:51:09.333426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:51:09.333426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:51:09.333426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:51:09.333426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:51:09.333426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:51:09.333426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:51:09.333426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:51:09.333426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 601/1195] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:51:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:51:09.338470 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74287c-be19-4080-b481-f7b6dfc0a374 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:51:09.430399 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74287c-be19-4080-b481-f7b6dfc0a374 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:51:09.461713 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3086992-d36c-4656-8083-7ac41889b56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:51:09.462386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3086992-d36c-4656-8083-7ac41889b56b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:09.462672 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3086992-d36c-4656-8083-7ac41889b56b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:09.462941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3086992-d36c-4656-8083-7ac41889b56b None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:09.463674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3086992-d36c-4656-8083-7ac41889b56b None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:51:09.463974 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3086992-d36c-4656-8083-7ac41889b56b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:51:09.465327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3086992-d36c-4656-8083-7ac41889b56b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:51:09.473386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 595/1196] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:51:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:51:09.478852 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b03c79-ac9b-4436-9546-b82acefff439 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:51:09.578781 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b03c79-ac9b-4436-9546-b82acefff439 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:51:09.609992 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7de93e-07a2-4b02-8065-54c94a7a1970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:51:09.611333 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7de93e-07a2-4b02-8065-54c94a7a1970 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:09.611612 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7de93e-07a2-4b02-8065-54c94a7a1970 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:09.611881 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7de93e-07a2-4b02-8065-54c94a7a1970 None admin] PATH_INFO: `/v3/users/455fce54edf64bb6bee0ad83a733a052` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:09.613043 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7de93e-07a2-4b02-8065-54c94a7a1970 None admin] RBAC: Authorizing `identity:get_user(user_id=455fce54edf64bb6bee0ad83a733a052)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:51:09.614497 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7de93e-07a2-4b02-8065-54c94a7a1970 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:51:09.616091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 602/1197] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 9 22:51:09 2022] GET /identity/v3/users/455fce54edf64bb6bee0ad83a733a052 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:51:09.621675 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3086992-d36c-4656-8083-7ac41889b56b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:51:09.713199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3086992-d36c-4656-8083-7ac41889b56b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:51:09.745150 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e55ce7-ee21-40e4-9046-4f4e58567657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:51:09.745783 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e55ce7-ee21-40e4-9046-4f4e58567657 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:09.746060 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e55ce7-ee21-40e4-9046-4f4e58567657 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:09.746372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e55ce7-ee21-40e4-9046-4f4e58567657 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:09.808064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e55ce7-ee21-40e4-9046-4f4e58567657 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:51:09.809483 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e55ce7-ee21-40e4-9046-4f4e58567657 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:51:09.900777 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 596/1198] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:51:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 22:51:09.906455 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7de93e-07a2-4b02-8065-54c94a7a1970 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:51:09.997401 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7de93e-07a2-4b02-8065-54c94a7a1970 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:51:10.030006 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8a00c-1932-4425-913b-bff4db94e14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:51:10.030860 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8a00c-1932-4425-913b-bff4db94e14d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:10.031136 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8a00c-1932-4425-913b-bff4db94e14d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:10.031405 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8a00c-1932-4425-913b-bff4db94e14d None admin] PATH_INFO: `/v3/projects/978da135c350476ba9cb078940ff0feb` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:10.032570 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8a00c-1932-4425-913b-bff4db94e14d None admin] RBAC: Authorizing `identity:get_project(project_id=978da135c350476ba9cb078940ff0feb)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:51:10.033979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8a00c-1932-4425-913b-bff4db94e14d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:51:10.035639 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 603/1199] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 9 22:51:09 2022] GET /identity/v3/projects/978da135c350476ba9cb078940ff0feb => generated 361 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:51:10.041380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e55ce7-ee21-40e4-9046-4f4e58567657 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:51:10.133293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e55ce7-ee21-40e4-9046-4f4e58567657 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:51:10.165139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bffba6-0183-4473-a5e0-b86819e7cc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:51:10.165715 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bffba6-0183-4473-a5e0-b86819e7cc0d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:51:10.165992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bffba6-0183-4473-a5e0-b86819e7cc0d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:51:10.166305 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bffba6-0183-4473-a5e0-b86819e7cc0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:51:10.167116 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bffba6-0183-4473-a5e0-b86819e7cc0d None admin] RBAC: Adding query filter params (role.id=bda550492f4f4c5c83f52fa89fc18a7d, scope.project.id=978da135c350476ba9cb078940ff0feb, user.id=455fce54edf64bb6bee0ad83a733a052) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:51:10.167424 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bffba6-0183-4473-a5e0-b86819e7cc0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:51:10.168842 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bffba6-0183-4473-a5e0-b86819e7cc0d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:51:10.174914 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 597/1200] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:51:10 2022] GET /identity/v3/role_assignments?user.id=455fce54edf64bb6bee0ad83a733a052&scope.project.id=978da135c350476ba9cb078940ff0feb&role.id=bda550492f4f4c5c83f52fa89fc18a7d => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:10.265566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de78512e-0998-4b03-9748-b79e4eb387eb None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:10.266297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de78512e-0998-4b03-9748-b79e4eb387eb None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:10.266297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de78512e-0998-4b03-9748-b79e4eb387eb None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:10.267709 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 604/1201] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:52:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:52:10.273402 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ebc57-20bf-4b5c-b6fa-1c04c7489d3a None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:10.273903 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ebc57-20bf-4b5c-b6fa-1c04c7489d3a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:10.274004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ebc57-20bf-4b5c-b6fa-1c04c7489d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:10.297529 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a32ebc57-20bf-4b5c-b6fa-1c04c7489d3a None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:52:10.298789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a32ebc57-20bf-4b5c-b6fa-1c04c7489d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:52:10.385259 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 598/1202] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:52:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:52:10.410884 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de78512e-0998-4b03-9748-b79e4eb387eb None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:10.534172 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de78512e-0998-4b03-9748-b79e4eb387eb None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:10.581457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184548bc-c10f-436a-a4cd-6d31f48381c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:10.584941 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184548bc-c10f-436a-a4cd-6d31f48381c3 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:10.585227 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184548bc-c10f-436a-a4cd-6d31f48381c3 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:10.585509 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184548bc-c10f-436a-a4cd-6d31f48381c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:10.597518 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184548bc-c10f-436a-a4cd-6d31f48381c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:10.599101 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184548bc-c10f-436a-a4cd-6d31f48381c3 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:10.623452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-184548bc-c10f-436a-a4cd-6d31f48381c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:10.623452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:10.623452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:10.623452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:10.623452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:10.623452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:10.623452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:10.623452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:10.623452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:10.623452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:10.623452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:10.623452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:10.623452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:52:10.623452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:52:10.623452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:52:10.625395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:52:10.625395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:10.625395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:10.625395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:10.625395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:10.625395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:10.625395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:10.625395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:10.625395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:52:10.625395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:10.625395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:10.625395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:10.625395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:10.625395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:10.625395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:10.625395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:10.625395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:10.626453 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:52:10.626453 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:52:10.626453 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:10.626453 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:52:10.626453 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 605/1203] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:52:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:10.632689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ebc57-20bf-4b5c-b6fa-1c04c7489d3a None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:10.771672 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ebc57-20bf-4b5c-b6fa-1c04c7489d3a None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:10.814553 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55754b3-9c3b-4f8b-9166-c0fca314b38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:10.815163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55754b3-9c3b-4f8b-9166-c0fca314b38b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:10.815507 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55754b3-9c3b-4f8b-9166-c0fca314b38b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:10.815822 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55754b3-9c3b-4f8b-9166-c0fca314b38b None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:10.816616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55754b3-9c3b-4f8b-9166-c0fca314b38b None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:10.816926 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55754b3-9c3b-4f8b-9166-c0fca314b38b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:10.818373 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55754b3-9c3b-4f8b-9166-c0fca314b38b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:10.833657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 599/1204] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:10.841431 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184548bc-c10f-436a-a4cd-6d31f48381c3 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:10.961010 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184548bc-c10f-436a-a4cd-6d31f48381c3 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:11.004403 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5eefc-f2a9-4b9b-a775-5d48b4cc00fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:11.005202 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5eefc-f2a9-4b9b-a775-5d48b4cc00fc None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:11.005606 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5eefc-f2a9-4b9b-a775-5d48b4cc00fc None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:11.005984 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5eefc-f2a9-4b9b-a775-5d48b4cc00fc None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:11.006890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5eefc-f2a9-4b9b-a775-5d48b4cc00fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:11.008482 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5eefc-f2a9-4b9b-a775-5d48b4cc00fc None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:11.042145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 606/1205] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 9 22:52:10 2022] POST /identity/v3/users => generated 251 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 22:52:13.483893 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb07d9-b2df-4839-96a0-ad61225cb848 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:13.484873 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb07d9-b2df-4839-96a0-ad61225cb848 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:13.485178 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb07d9-b2df-4839-96a0-ad61225cb848 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:13.486800 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 600/1206] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:52:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:52:13.492344 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e36675-263d-4e59-aa6a-9c4e97c4a54c None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:13.492903 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e36675-263d-4e59-aa6a-9c4e97c4a54c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:13.492903 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e36675-263d-4e59-aa6a-9c4e97c4a54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:13.510308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-d0e36675-263d-4e59-aa6a-9c4e97c4a54c None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:52:13.511627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d0e36675-263d-4e59-aa6a-9c4e97c4a54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:52:13.585487 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 607/1207] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:52:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:52:13.612284 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb07d9-b2df-4839-96a0-ad61225cb848 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:13.729402 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb07d9-b2df-4839-96a0-ad61225cb848 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:13.771505 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d5b0e-d15a-4b4e-b33e-9b29b31df9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:13.772224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d5b0e-d15a-4b4e-b33e-9b29b31df9e6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:13.772505 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d5b0e-d15a-4b4e-b33e-9b29b31df9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:13.772790 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d5b0e-d15a-4b4e-b33e-9b29b31df9e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:13.783732 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d5b0e-d15a-4b4e-b33e-9b29b31df9e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:52:13.791506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-d93d5b0e-d15a-4b4e-b33e-9b29b31df9e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 22:52:13.791506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:13.791506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:13.791506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:13.791506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:13.791506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:13.791506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:13.791506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:13.791506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:13.791506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:13.791506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:13.791506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:13.791506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:52:13.791506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:52:13.791506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:52:13.791506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:52:13.792674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:52:13.792674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:52:13.792674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:52:13.792674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:52:13.792674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:13.792674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:13.792674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:13.792674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:13.792674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:13.792674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:13.792674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:13.792674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:52:13.792674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:13.792674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:13.792674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:13.792674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:13.793539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:13.793539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:13.793539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:13.793539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:13.793539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:52:13.793539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:52:13.793539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:52:13.793539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:52:13.793539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:52:13.793539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:52:13.793539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 22:52:13.793539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:52:13.793539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 601/1208] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 9 22:52:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:13.799074 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e36675-263d-4e59-aa6a-9c4e97c4a54c None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:13.901674 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e36675-263d-4e59-aa6a-9c4e97c4a54c None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:13.936016 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5a21e-fa6d-468e-944a-45b2e9b55eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:13.937348 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5a21e-fa6d-468e-944a-45b2e9b55eac None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:13.937682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5a21e-fa6d-468e-944a-45b2e9b55eac None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:13.938067 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5a21e-fa6d-468e-944a-45b2e9b55eac None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:13.939399 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5a21e-fa6d-468e-944a-45b2e9b55eac None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:13.939718 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5a21e-fa6d-468e-944a-45b2e9b55eac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:13.941169 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5a21e-fa6d-468e-944a-45b2e9b55eac None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:13.959613 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 608/1209] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 9 22:52:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:13.961410 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d5b0e-d15a-4b4e-b33e-9b29b31df9e6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:14.075442 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d5b0e-d15a-4b4e-b33e-9b29b31df9e6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:14.110174 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c3315-da12-4711-bf3a-cf636b178b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:14.110962 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c3315-da12-4711-bf3a-cf636b178b05 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:14.111243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c3315-da12-4711-bf3a-cf636b178b05 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:14.111563 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c3315-da12-4711-bf3a-cf636b178b05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:14.121715 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c3315-da12-4711-bf3a-cf636b178b05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:14.123230 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c3315-da12-4711-bf3a-cf636b178b05 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:14.141874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-848c3315-da12-4711-bf3a-cf636b178b05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:14.141874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:14.141874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:14.141874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:14.141874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:14.141874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:14.141874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:14.141874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:14.141874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:14.141874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:14.141874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:14.141874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:14.141874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:52:14.141874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:52:14.141874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:52:14.143032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:52:14.143032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:14.143032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:14.143032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:14.143032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:14.143032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:14.143032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:14.143032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:14.143032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:52:14.143032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:14.143032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:14.143032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:14.143032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:14.143032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:14.143032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:14.143032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:14.143032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:14.144135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:52:14.144135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:52:14.144135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:14.144135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:52:14.145350 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 602/1210] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:52:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:14.151264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5a21e-fa6d-468e-944a-45b2e9b55eac None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:14.259304 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5a21e-fa6d-468e-944a-45b2e9b55eac None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:14.295644 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678f8e0-78f4-47a5-bf31-619ca18d4ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:14.297084 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678f8e0-78f4-47a5-bf31-619ca18d4ab7 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:14.297456 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678f8e0-78f4-47a5-bf31-619ca18d4ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:14.297831 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678f8e0-78f4-47a5-bf31-619ca18d4ab7 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:14.298699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678f8e0-78f4-47a5-bf31-619ca18d4ab7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:14.299078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678f8e0-78f4-47a5-bf31-619ca18d4ab7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:14.300546 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678f8e0-78f4-47a5-bf31-619ca18d4ab7 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:14.315536 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 609/1211] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:52:14.321143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c3315-da12-4711-bf3a-cf636b178b05 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:14.426013 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c3315-da12-4711-bf3a-cf636b178b05 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:14.461174 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390abf2-7b08-468d-930f-f50b590d00b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:14.462033 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390abf2-7b08-468d-930f-f50b590d00b8 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:14.462388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390abf2-7b08-468d-930f-f50b590d00b8 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:14.462646 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390abf2-7b08-468d-930f-f50b590d00b8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:14.470414 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390abf2-7b08-468d-930f-f50b590d00b8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:14.471851 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390abf2-7b08-468d-930f-f50b590d00b8 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:14.489130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-2390abf2-7b08-468d-930f-f50b590d00b8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 22:52:14.489130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:14.489130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:14.489130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:14.489130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:14.489130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:14.489130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:14.489130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:14.489130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:14.489130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:14.489130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:14.489130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:14.489130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:52:14.489130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:52:14.489130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:52:14.489130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:52:14.490778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:14.490778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:14.490778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:52:14.490778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:52:14.490778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:52:14.490778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:52:14.490778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:14.490778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:14.490778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:14.490778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:14.490778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:14.490778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:52:14.490778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:14.490778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:14.490778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:14.490778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:14.490778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:14.490778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:14.491998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:14.491998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:14.491998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:52:14.491998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:52:14.491998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:52:14.491998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:52:14.491998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:52:14.491998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:52:14.491998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 22:52:14.491998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:52:14.495161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 603/1212] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:52:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:14.496402 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678f8e0-78f4-47a5-bf31-619ca18d4ab7 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:14.599958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678f8e0-78f4-47a5-bf31-619ca18d4ab7 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:14.639680 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9485e6-d46e-4c6d-b5f2-5a7b36556360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:14.640486 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9485e6-d46e-4c6d-b5f2-5a7b36556360 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:14.640758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9485e6-d46e-4c6d-b5f2-5a7b36556360 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:14.641055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9485e6-d46e-4c6d-b5f2-5a7b36556360 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:14.641967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9485e6-d46e-4c6d-b5f2-5a7b36556360 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:14.642351 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9485e6-d46e-4c6d-b5f2-5a7b36556360 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:14.643798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9485e6-d46e-4c6d-b5f2-5a7b36556360 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:14.662895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 610/1213] 10.222.0.23 () {56 vars in 1143 bytes} [Mon May 9 22:52:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:14.668235 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390abf2-7b08-468d-930f-f50b590d00b8 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:14.780384 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390abf2-7b08-468d-930f-f50b590d00b8 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:14.817621 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1912b-f999-475c-bd74-d2c0a3a6c435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:14.818302 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1912b-f999-475c-bd74-d2c0a3a6c435 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:14.818581 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1912b-f999-475c-bd74-d2c0a3a6c435 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:14.818849 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1912b-f999-475c-bd74-d2c0a3a6c435 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:14.890691 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1912b-f999-475c-bd74-d2c0a3a6c435 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:14.892158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1912b-f999-475c-bd74-d2c0a3a6c435 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:14.994847 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 604/1214] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:52:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 22:52:15.000552 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9485e6-d46e-4c6d-b5f2-5a7b36556360 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:15.104518 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9485e6-d46e-4c6d-b5f2-5a7b36556360 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:15.143318 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bdfe9e-dbae-4e43-b485-4e5621ae02cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:15.144126 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bdfe9e-dbae-4e43-b485-4e5621ae02cb None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:15.144492 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bdfe9e-dbae-4e43-b485-4e5621ae02cb None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:15.144710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bdfe9e-dbae-4e43-b485-4e5621ae02cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:15.155209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bdfe9e-dbae-4e43-b485-4e5621ae02cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:15.156672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bdfe9e-dbae-4e43-b485-4e5621ae02cb None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:15.174717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-97bdfe9e-dbae-4e43-b485-4e5621ae02cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:15.174717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:15.174717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:15.174717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:15.174717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:15.174717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:15.174717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:15.174717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:15.174717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:15.174717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:15.174717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:15.174717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:15.174717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:52:15.174717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:52:15.174717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:52:15.180369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:52:15.180369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:15.180369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:15.180369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:15.180369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:15.180369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:15.180369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:15.180369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:15.180369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:52:15.180369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:15.180369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:15.180369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:15.180369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:15.180369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:15.180369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:15.180369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:15.180369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:15.181278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:52:15.181278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:52:15.181278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:15.181278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:52:15.181278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 611/1215] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:52:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:15.181650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1912b-f999-475c-bd74-d2c0a3a6c435 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:15.280854 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1912b-f999-475c-bd74-d2c0a3a6c435 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:15.315388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ed7b0-e0ea-4ee4-b2d4-ee696f541592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:15.316040 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ed7b0-e0ea-4ee4-b2d4-ee696f541592 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:15.316318 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ed7b0-e0ea-4ee4-b2d4-ee696f541592 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:15.316673 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ed7b0-e0ea-4ee4-b2d4-ee696f541592 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:15.317399 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ed7b0-e0ea-4ee4-b2d4-ee696f541592 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:15.317699 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ed7b0-e0ea-4ee4-b2d4-ee696f541592 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:15.319189 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ed7b0-e0ea-4ee4-b2d4-ee696f541592 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:15.331883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 605/1216] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:52:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:15.338054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bdfe9e-dbae-4e43-b485-4e5621ae02cb None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:15.438739 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bdfe9e-dbae-4e43-b485-4e5621ae02cb None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:15.473317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097554e-8e7a-4fb7-ac7d-c307fde2b8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:15.474153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097554e-8e7a-4fb7-ac7d-c307fde2b8be None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:15.475437 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097554e-8e7a-4fb7-ac7d-c307fde2b8be None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:15.475437 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097554e-8e7a-4fb7-ac7d-c307fde2b8be None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:15.481236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097554e-8e7a-4fb7-ac7d-c307fde2b8be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:15.482698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097554e-8e7a-4fb7-ac7d-c307fde2b8be None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:15.496871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-1097554e-8e7a-4fb7-ac7d-c307fde2b8be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:52:15.496871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:15.496871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:15.496871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:15.496871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:15.496871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:15.496871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:15.496871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:15.496871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:15.496871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:15.496871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:15.496871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:15.496871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:52:15.496871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:52:15.496871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:52:15.498361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:52:15.498361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:15.498361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:15.498361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:15.498361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:15.498361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:15.498361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:15.498361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:15.498361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:52:15.498361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:15.498361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:15.498361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:15.498361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:15.498361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:15.498361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:15.498361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:15.498361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:15.499885 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:52:15.499885 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:52:15.499885 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:52:15.499885 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:52:15.499885 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:52:15.499885 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:52:15.499885 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:52:15.499885 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:52:15.499885 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 612/1217] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:52:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:15.504534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ed7b0-e0ea-4ee4-b2d4-ee696f541592 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:15.607079 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ed7b0-e0ea-4ee4-b2d4-ee696f541592 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:15.646571 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5474b4-c19a-4a43-b1cd-18a1a1d40dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:15.647251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5474b4-c19a-4a43-b1cd-18a1a1d40dda None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:15.647533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5474b4-c19a-4a43-b1cd-18a1a1d40dda None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:15.647806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5474b4-c19a-4a43-b1cd-18a1a1d40dda None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:15.648699 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5474b4-c19a-4a43-b1cd-18a1a1d40dda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:15.649032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5474b4-c19a-4a43-b1cd-18a1a1d40dda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:15.650493 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5474b4-c19a-4a43-b1cd-18a1a1d40dda None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:15.667462 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 606/1218] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:52:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:15.669198 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097554e-8e7a-4fb7-ac7d-c307fde2b8be None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:15.776797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097554e-8e7a-4fb7-ac7d-c307fde2b8be None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:15.813304 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a57ebc-601d-4feb-8b97-15f05791c6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:15.813964 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a57ebc-601d-4feb-8b97-15f05791c6a8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:15.814360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a57ebc-601d-4feb-8b97-15f05791c6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:15.814644 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a57ebc-601d-4feb-8b97-15f05791c6a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:15.815510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a57ebc-601d-4feb-8b97-15f05791c6a8 None admin] RBAC: Adding query filter params (role.id=44847d50e6cb4bc7938c59b4e2d99833, scope.project.id=0500e378ea374ca692e230bbd8781318, user.id=cf9bc2fa959d4706b9f9e87509829059) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:15.815820 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a57ebc-601d-4feb-8b97-15f05791c6a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:15.817285 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a57ebc-601d-4feb-8b97-15f05791c6a8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:15.822931 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 613/1219] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:52:15 2022] GET /identity/v3/role_assignments?user.id=cf9bc2fa959d4706b9f9e87509829059&scope.project.id=0500e378ea374ca692e230bbd8781318&role.id=44847d50e6cb4bc7938c59b4e2d99833 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:18.069195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e4f79-b97c-4e5e-b676-288074183f6b None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:18.069773 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e4f79-b97c-4e5e-b676-288074183f6b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:18.069773 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e4f79-b97c-4e5e-b676-288074183f6b None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:18.071143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 607/1220] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:52:18.076529 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd04cf3-336b-49e2-98a5-dc7660b1b6ba None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:18.076979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd04cf3-336b-49e2-98a5-dc7660b1b6ba None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:18.077102 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd04cf3-336b-49e2-98a5-dc7660b1b6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:18.094506 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-4fd04cf3-336b-49e2-98a5-dc7660b1b6ba None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:52:18.095689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4fd04cf3-336b-49e2-98a5-dc7660b1b6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:52:18.166163 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 614/1221] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:52:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:52:18.189244 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e4f79-b97c-4e5e-b676-288074183f6b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:18.301357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e4f79-b97c-4e5e-b676-288074183f6b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:18.337269 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea68ef-32af-417b-b7ee-321aa54da822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:18.338020 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea68ef-32af-417b-b7ee-321aa54da822 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:18.338350 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea68ef-32af-417b-b7ee-321aa54da822 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:18.338580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea68ef-32af-417b-b7ee-321aa54da822 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:18.349562 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea68ef-32af-417b-b7ee-321aa54da822 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:52:18.357236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-7aea68ef-32af-417b-b7ee-321aa54da822 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 22:52:18.357236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:18.357236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:18.357236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:18.357236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:18.357236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:18.357236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:18.357236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:18.357236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:18.357236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:18.357236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:18.357236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:18.357236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:52:18.357236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:52:18.357236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:52:18.357236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:52:18.358818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:52:18.358818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:52:18.358818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:52:18.358818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:52:18.358818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:18.358818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:18.358818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:18.358818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:18.358818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:18.358818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:18.358818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:18.358818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:52:18.358818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:18.358818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:18.358818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:18.358818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:18.359976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:18.359976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:18.359976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:18.359976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:18.359976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:52:18.359976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:52:18.359976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:52:18.359976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:52:18.359976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:52:18.359976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:52:18.359976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 22:52:18.359976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:52:18.366144 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 608/1222] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 9 22:52:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:18.366297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd04cf3-336b-49e2-98a5-dc7660b1b6ba None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:18.474350 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd04cf3-336b-49e2-98a5-dc7660b1b6ba None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:18.510848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf5b870-e6cf-413a-bc7a-edb80e8bd690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:18.511517 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf5b870-e6cf-413a-bc7a-edb80e8bd690 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:18.511807 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf5b870-e6cf-413a-bc7a-edb80e8bd690 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:18.512080 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf5b870-e6cf-413a-bc7a-edb80e8bd690 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:18.512821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf5b870-e6cf-413a-bc7a-edb80e8bd690 None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:18.513123 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf5b870-e6cf-413a-bc7a-edb80e8bd690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:18.514578 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf5b870-e6cf-413a-bc7a-edb80e8bd690 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:18.522555 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 615/1223] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 9 22:52:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:18.528269 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea68ef-32af-417b-b7ee-321aa54da822 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:18.632409 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea68ef-32af-417b-b7ee-321aa54da822 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:18.666866 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5994182-197c-4a5d-bb35-5af95a8b6fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:18.667654 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5994182-197c-4a5d-bb35-5af95a8b6fdb None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:18.667935 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5994182-197c-4a5d-bb35-5af95a8b6fdb None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:18.668205 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5994182-197c-4a5d-bb35-5af95a8b6fdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:18.679444 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5994182-197c-4a5d-bb35-5af95a8b6fdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:18.680892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5994182-197c-4a5d-bb35-5af95a8b6fdb None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:18.698387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-d5994182-197c-4a5d-bb35-5af95a8b6fdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:18.698387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:18.698387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:18.698387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:18.698387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:18.698387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:18.698387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:18.698387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:18.698387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:18.698387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:18.698387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:18.698387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:18.698387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:52:18.698387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:52:18.698387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:52:18.699441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:52:18.699441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:18.699441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:18.699441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:18.699441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:18.699441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:18.699441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:18.699441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:18.699441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:52:18.699441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:18.699441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:18.699441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:18.699441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:18.699441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:18.699441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:18.699441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:18.699441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:18.700747 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:52:18.700747 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:52:18.700747 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:18.700747 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:52:18.700747 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 609/1224] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:52:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:18.705423 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf5b870-e6cf-413a-bc7a-edb80e8bd690 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:18.812262 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf5b870-e6cf-413a-bc7a-edb80e8bd690 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:18.845807 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a5bdc-7887-4bc7-b14e-ece3114b4700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:18.846523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a5bdc-7887-4bc7-b14e-ece3114b4700 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:18.846808 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a5bdc-7887-4bc7-b14e-ece3114b4700 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:18.847116 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a5bdc-7887-4bc7-b14e-ece3114b4700 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:18.847805 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5a5bdc-7887-4bc7-b14e-ece3114b4700 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:18.848113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5a5bdc-7887-4bc7-b14e-ece3114b4700 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:18.849471 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5a5bdc-7887-4bc7-b14e-ece3114b4700 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:18.862813 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 616/1225] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:52:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:18.868526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5994182-197c-4a5d-bb35-5af95a8b6fdb None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:18.971709 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5994182-197c-4a5d-bb35-5af95a8b6fdb None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:19.006085 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ef02f-f930-44f1-b4f7-dbeafd3edcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:19.007025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ef02f-f930-44f1-b4f7-dbeafd3edcce None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:19.007371 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ef02f-f930-44f1-b4f7-dbeafd3edcce None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:19.007681 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ef02f-f930-44f1-b4f7-dbeafd3edcce None admin] PATH_INFO: `/v3/users/placement` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:19.015861 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ef02f-f930-44f1-b4f7-dbeafd3edcce None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:19.017292 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ef02f-f930-44f1-b4f7-dbeafd3edcce None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:19.031458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-e18ef02f-f930-44f1-b4f7-dbeafd3edcce None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 22:52:19.031458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:19.031458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:19.031458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:19.031458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:19.031458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:19.031458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:19.031458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:19.031458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:19.031458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:19.031458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:19.031458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:19.031458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:52:19.031458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:52:19.031458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:52:19.031458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:52:19.032463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:19.032463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:19.032463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:52:19.032463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:52:19.032463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:52:19.032463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:52:19.032463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:19.032463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:19.032463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:19.032463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:19.032463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:19.032463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:52:19.032463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:19.032463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:19.032463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:19.032463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:19.032463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:19.032463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:19.038357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:19.038357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:19.038357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:52:19.038357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:52:19.038357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:52:19.038357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:52:19.038357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:52:19.038357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:52:19.038357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 22:52:19.038357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:52:19.038357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 610/1226] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:52:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:19.040317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a5bdc-7887-4bc7-b14e-ece3114b4700 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:19.156787 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a5bdc-7887-4bc7-b14e-ece3114b4700 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:19.198218 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37e8f1-8fec-4ef0-abfb-ce8cfdb6f136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:19.199029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37e8f1-8fec-4ef0-abfb-ce8cfdb6f136 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:19.199373 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37e8f1-8fec-4ef0-abfb-ce8cfdb6f136 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:19.199666 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37e8f1-8fec-4ef0-abfb-ce8cfdb6f136 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:19.200761 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d37e8f1-8fec-4ef0-abfb-ce8cfdb6f136 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:19.201140 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d37e8f1-8fec-4ef0-abfb-ce8cfdb6f136 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:19.203013 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d37e8f1-8fec-4ef0-abfb-ce8cfdb6f136 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:19.225041 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 617/1227] 10.222.0.23 () {56 vars in 1143 bytes} [Mon May 9 22:52:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:19.231211 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ef02f-f930-44f1-b4f7-dbeafd3edcce None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:19.343985 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ef02f-f930-44f1-b4f7-dbeafd3edcce None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:19.380260 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201c660-be08-4097-b0e9-57afad299194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:19.381049 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201c660-be08-4097-b0e9-57afad299194 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:19.381329 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201c660-be08-4097-b0e9-57afad299194 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:19.381608 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201c660-be08-4097-b0e9-57afad299194 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:19.392203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0201c660-be08-4097-b0e9-57afad299194 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:19.393648 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0201c660-be08-4097-b0e9-57afad299194 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:19.410100 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-0201c660-be08-4097-b0e9-57afad299194 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:19.410100 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:19.410100 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:19.410100 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:19.410100 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:19.410100 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:19.410100 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:19.410100 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:19.410100 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:19.410100 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:19.410100 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:19.410100 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:19.410100 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:52:19.410100 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:52:19.410100 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:52:19.411212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:52:19.411212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:19.411212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:19.411212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:19.411212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:19.411212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:19.411212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:19.411212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:19.411212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:52:19.411212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:19.411212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:19.411212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:19.411212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:19.411212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:19.411212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:19.411212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:19.411212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:19.412225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:52:19.412225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:52:19.412225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:19.412225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:52:19.412225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 611/1228] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:52:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:19.417095 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37e8f1-8fec-4ef0-abfb-ce8cfdb6f136 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:19.534416 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37e8f1-8fec-4ef0-abfb-ce8cfdb6f136 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:19.570226 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcf605-47e5-49b0-bc7a-ba42b3e118e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:19.570899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcf605-47e5-49b0-bc7a-ba42b3e118e4 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:19.571185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcf605-47e5-49b0-bc7a-ba42b3e118e4 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:19.571459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcf605-47e5-49b0-bc7a-ba42b3e118e4 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:19.572170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcf605-47e5-49b0-bc7a-ba42b3e118e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:19.572474 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcf605-47e5-49b0-bc7a-ba42b3e118e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:19.573846 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcf605-47e5-49b0-bc7a-ba42b3e118e4 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:19.585031 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 618/1229] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:19.590756 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201c660-be08-4097-b0e9-57afad299194 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:19.696892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201c660-be08-4097-b0e9-57afad299194 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:19.730803 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815403b-3925-4eb4-bb93-e5582d8d1b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:19.732585 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815403b-3925-4eb4-bb93-e5582d8d1b1c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:19.732913 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815403b-3925-4eb4-bb93-e5582d8d1b1c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:19.733190 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815403b-3925-4eb4-bb93-e5582d8d1b1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:19.741223 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815403b-3925-4eb4-bb93-e5582d8d1b1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:19.742731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815403b-3925-4eb4-bb93-e5582d8d1b1c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:19.759001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-2815403b-3925-4eb4-bb93-e5582d8d1b1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:52:19.759001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:19.759001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:19.759001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:19.759001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:19.759001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:19.759001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:19.759001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:19.759001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:19.759001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:19.759001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:19.759001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:19.759001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:52:19.759001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:52:19.759001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:52:19.760029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:52:19.760029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:19.760029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:19.760029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:19.760029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:19.760029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:19.760029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:19.760029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:19.760029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:52:19.760029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:19.760029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:19.760029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:19.760029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:19.760029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:19.760029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:19.760029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:19.760029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:19.765863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:52:19.765863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:52:19.765863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:52:19.765863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:52:19.765863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:52:19.765863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:52:19.765863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:52:19.765863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:52:19.765863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 612/1230] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:52:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:19.766635 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcf605-47e5-49b0-bc7a-ba42b3e118e4 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:19.876964 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcf605-47e5-49b0-bc7a-ba42b3e118e4 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:19.909879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e6a44-9bad-4a84-8149-04a3cd8e8440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:19.910616 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e6a44-9bad-4a84-8149-04a3cd8e8440 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:19.910994 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e6a44-9bad-4a84-8149-04a3cd8e8440 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:19.911328 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e6a44-9bad-4a84-8149-04a3cd8e8440 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:19.912179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177e6a44-9bad-4a84-8149-04a3cd8e8440 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:19.912522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177e6a44-9bad-4a84-8149-04a3cd8e8440 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:19.914108 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177e6a44-9bad-4a84-8149-04a3cd8e8440 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:19.928105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 619/1231] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:52:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:52:19.934315 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815403b-3925-4eb4-bb93-e5582d8d1b1c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:20.033831 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815403b-3925-4eb4-bb93-e5582d8d1b1c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:20.067052 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec2bf4-ed17-438b-8b51-2ed04b4ec7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:20.067889 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec2bf4-ed17-438b-8b51-2ed04b4ec7c6 None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:20.068150 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec2bf4-ed17-438b-8b51-2ed04b4ec7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:20.068423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec2bf4-ed17-438b-8b51-2ed04b4ec7c6 None admin] PATH_INFO: `/v3/projects/0500e378ea374ca692e230bbd8781318/users/cf9bc2fa959d4706b9f9e87509829059/roles/44847d50e6cb4bc7938c59b4e2d99833` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:20.083394 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec2bf4-ed17-438b-8b51-2ed04b4ec7c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=0500e378ea374ca692e230bbd8781318, user_id=cf9bc2fa959d4706b9f9e87509829059, role_id=44847d50e6cb4bc7938c59b4e2d99833)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:20.084819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec2bf4-ed17-438b-8b51-2ed04b4ec7c6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:20.097017 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 613/1232] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:52:19 2022] PUT /identity/v3/projects/0500e378ea374ca692e230bbd8781318/users/cf9bc2fa959d4706b9f9e87509829059/roles/44847d50e6cb4bc7938c59b4e2d99833 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:52:22.565337 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa11c8-c07d-4b09-887e-17c72c1e424e None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:22.565873 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa11c8-c07d-4b09-887e-17c72c1e424e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:22.565873 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa11c8-c07d-4b09-887e-17c72c1e424e None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:22.567432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 620/1233] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:52:22.572616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb43118-1bbe-4d98-817f-a9f96d1f1afa None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:22.573025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb43118-1bbe-4d98-817f-a9f96d1f1afa None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:22.573167 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb43118-1bbe-4d98-817f-a9f96d1f1afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:22.592001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-2cb43118-1bbe-4d98-817f-a9f96d1f1afa None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:52:22.593761 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2cb43118-1bbe-4d98-817f-a9f96d1f1afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:52:22.678708 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 614/1234] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:52:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:52:22.702887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa11c8-c07d-4b09-887e-17c72c1e424e None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:22.813821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa11c8-c07d-4b09-887e-17c72c1e424e None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:22.851982 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce0584-88bf-4420-a31b-d5d040b5cf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:22.852792 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce0584-88bf-4420-a31b-d5d040b5cf8e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:22.853091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce0584-88bf-4420-a31b-d5d040b5cf8e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:22.853366 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce0584-88bf-4420-a31b-d5d040b5cf8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:22.866274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce0584-88bf-4420-a31b-d5d040b5cf8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:52:22.874691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-21ce0584-88bf-4420-a31b-d5d040b5cf8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 22:52:22.874691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:22.874691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:22.874691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:22.874691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:22.874691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:22.874691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:22.874691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:22.874691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:22.874691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:22.874691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:22.874691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:22.874691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:52:22.874691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:52:22.874691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:52:22.874691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:52:22.875699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:52:22.875699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:52:22.875699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:52:22.875699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:52:22.875699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:22.875699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:22.875699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:22.875699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:22.875699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:22.875699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:22.875699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:22.875699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:52:22.875699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:22.875699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:22.875699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:22.875699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:22.876597 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:22.876597 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:22.876597 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:22.876597 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:22.876597 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:52:22.876597 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:52:22.876597 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:52:22.876597 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:52:22.876597 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:52:22.876597 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:52:22.876597 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 22:52:22.876597 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:52:22.876597 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 621/1235] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 9 22:52:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 22:52:22.882198 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb43118-1bbe-4d98-817f-a9f96d1f1afa None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:22.991795 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb43118-1bbe-4d98-817f-a9f96d1f1afa None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:23.032749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d20184-bd14-40c6-aac3-530d458c5112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:23.033421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d20184-bd14-40c6-aac3-530d458c5112 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:23.033719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d20184-bd14-40c6-aac3-530d458c5112 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:23.033972 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d20184-bd14-40c6-aac3-530d458c5112 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:23.034980 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d20184-bd14-40c6-aac3-530d458c5112 None admin] RBAC: Adding query filter params (name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:23.035404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d20184-bd14-40c6-aac3-530d458c5112 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:23.037144 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d20184-bd14-40c6-aac3-530d458c5112 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:23.046941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 615/1236] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 9 22:52:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:23.052901 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce0584-88bf-4420-a31b-d5d040b5cf8e None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:23.161216 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce0584-88bf-4420-a31b-d5d040b5cf8e None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:23.193905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5363f-d24a-4daa-bf37-0f40a2bd6716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:23.194838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5363f-d24a-4daa-bf37-0f40a2bd6716 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:23.195120 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5363f-d24a-4daa-bf37-0f40a2bd6716 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:23.195453 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5363f-d24a-4daa-bf37-0f40a2bd6716 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:23.207079 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5363f-d24a-4daa-bf37-0f40a2bd6716 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:23.208518 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5363f-d24a-4daa-bf37-0f40a2bd6716 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:23.225834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-c6a5363f-d24a-4daa-bf37-0f40a2bd6716 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:23.225834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:23.225834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:23.225834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:23.225834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:23.225834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:23.225834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:23.225834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:23.225834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:23.225834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:23.225834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:23.225834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:23.225834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:52:23.225834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:52:23.225834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:52:23.226848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:52:23.226848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:23.226848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:23.226848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:23.226848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:23.226848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:23.226848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:23.226848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:23.226848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:52:23.226848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:23.226848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:23.226848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:23.226848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:23.226848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:23.226848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:23.226848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:23.226848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:23.227786 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:52:23.227786 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:52:23.227786 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:23.227786 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:52:23.227786 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 622/1237] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:52:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:23.233034 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d20184-bd14-40c6-aac3-530d458c5112 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:23.337931 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d20184-bd14-40c6-aac3-530d458c5112 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:23.377420 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73107d1-84ad-4bca-8844-c6052ec0c8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:23.378086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73107d1-84ad-4bca-8844-c6052ec0c8d6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:23.378533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73107d1-84ad-4bca-8844-c6052ec0c8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:23.378810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73107d1-84ad-4bca-8844-c6052ec0c8d6 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:23.379530 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73107d1-84ad-4bca-8844-c6052ec0c8d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:23.379834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73107d1-84ad-4bca-8844-c6052ec0c8d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:23.381230 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73107d1-84ad-4bca-8844-c6052ec0c8d6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:23.393854 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 616/1238] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:23.399547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5363f-d24a-4daa-bf37-0f40a2bd6716 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:23.508225 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5363f-d24a-4daa-bf37-0f40a2bd6716 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:23.543160 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392e014-a947-4265-b97f-024dbe06326f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:23.544064 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392e014-a947-4265-b97f-024dbe06326f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:23.544345 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392e014-a947-4265-b97f-024dbe06326f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:23.544618 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392e014-a947-4265-b97f-024dbe06326f None admin] PATH_INFO: `/v3/users/placement` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:23.553197 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392e014-a947-4265-b97f-024dbe06326f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:23.554681 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392e014-a947-4265-b97f-024dbe06326f None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:23.569721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-1392e014-a947-4265-b97f-024dbe06326f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 22:52:23.569721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:23.569721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:23.569721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:23.569721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:23.569721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:23.569721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:23.569721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:23.569721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:23.569721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:23.569721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:23.569721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:23.569721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:52:23.569721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:52:23.569721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:52:23.569721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:52:23.571414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:23.571414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:23.571414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:52:23.571414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:52:23.571414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:52:23.571414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:52:23.571414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:23.571414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:23.571414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:23.571414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:23.571414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:23.571414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:52:23.571414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:23.571414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:23.571414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:23.571414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:23.571414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:23.571414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:23.572365 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:23.572365 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:23.572365 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:52:23.572365 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:52:23.572365 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:52:23.572365 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:52:23.572365 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:52:23.572365 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:52:23.572365 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 22:52:23.572365 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:52:23.572365 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 623/1239] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:52:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:23.576880 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73107d1-84ad-4bca-8844-c6052ec0c8d6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:23.687367 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73107d1-84ad-4bca-8844-c6052ec0c8d6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:23.724472 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5280e1-d12e-406b-a036-691841d992be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:23.725151 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5280e1-d12e-406b-a036-691841d992be None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:23.725436 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5280e1-d12e-406b-a036-691841d992be None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:23.725709 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5280e1-d12e-406b-a036-691841d992be None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:23.726586 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5280e1-d12e-406b-a036-691841d992be None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:23.726892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5280e1-d12e-406b-a036-691841d992be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:23.728325 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5280e1-d12e-406b-a036-691841d992be None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:23.747718 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 617/1240] 10.222.0.23 () {56 vars in 1143 bytes} [Mon May 9 22:52:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:23.753679 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392e014-a947-4265-b97f-024dbe06326f None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:23.860367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392e014-a947-4265-b97f-024dbe06326f None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:23.896046 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fd249-dbf8-4a98-8a9c-24f23e6a35af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:23.896857 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fd249-dbf8-4a98-8a9c-24f23e6a35af None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:23.897244 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fd249-dbf8-4a98-8a9c-24f23e6a35af None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:23.897522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fd249-dbf8-4a98-8a9c-24f23e6a35af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:23.966352 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70fd249-dbf8-4a98-8a9c-24f23e6a35af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:23.968453 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70fd249-dbf8-4a98-8a9c-24f23e6a35af None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:24.069304 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 624/1241] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:52:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 22:52:24.076231 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5280e1-d12e-406b-a036-691841d992be None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:24.177925 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5280e1-d12e-406b-a036-691841d992be None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:24.215510 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1f6b1-25e6-45f1-bf72-b5aaae3b95a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:24.216508 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1f6b1-25e6-45f1-bf72-b5aaae3b95a0 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:24.216759 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1f6b1-25e6-45f1-bf72-b5aaae3b95a0 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:24.217065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1f6b1-25e6-45f1-bf72-b5aaae3b95a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:24.227644 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1f6b1-25e6-45f1-bf72-b5aaae3b95a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:24.229054 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1f6b1-25e6-45f1-bf72-b5aaae3b95a0 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:24.246412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-6ee1f6b1-25e6-45f1-bf72-b5aaae3b95a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:24.246412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:24.246412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:24.246412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:24.246412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:24.246412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:24.246412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:24.246412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:24.246412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:24.246412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:24.246412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:24.246412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:24.246412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:52:24.246412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:52:24.246412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:52:24.247387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:52:24.247387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:24.247387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:24.247387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:24.247387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:24.247387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:24.247387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:24.247387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:24.247387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:52:24.247387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:24.247387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:24.247387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:24.247387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:24.247387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:24.247387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:24.247387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:24.247387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:24.252889 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:52:24.252889 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:52:24.252889 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:24.252889 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:52:24.252889 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 618/1242] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:52:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:24.254239 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fd249-dbf8-4a98-8a9c-24f23e6a35af None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:24.359172 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fd249-dbf8-4a98-8a9c-24f23e6a35af None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:24.398447 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68ea67-bd99-412c-9a20-2461f725be75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:24.399230 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68ea67-bd99-412c-9a20-2461f725be75 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:24.399590 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68ea67-bd99-412c-9a20-2461f725be75 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:24.399913 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68ea67-bd99-412c-9a20-2461f725be75 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:24.400821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68ea67-bd99-412c-9a20-2461f725be75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:24.401224 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68ea67-bd99-412c-9a20-2461f725be75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:24.402882 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68ea67-bd99-412c-9a20-2461f725be75 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:24.415182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 625/1243] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:52:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:24.421264 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1f6b1-25e6-45f1-bf72-b5aaae3b95a0 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:24.529131 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1f6b1-25e6-45f1-bf72-b5aaae3b95a0 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:24.563938 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927739e-4eee-4072-b7bb-7b4bfbde95bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:24.564720 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927739e-4eee-4072-b7bb-7b4bfbde95bf None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:24.565000 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927739e-4eee-4072-b7bb-7b4bfbde95bf None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:24.565271 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927739e-4eee-4072-b7bb-7b4bfbde95bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:24.571827 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6927739e-4eee-4072-b7bb-7b4bfbde95bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:24.573258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6927739e-4eee-4072-b7bb-7b4bfbde95bf None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:24.587618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-6927739e-4eee-4072-b7bb-7b4bfbde95bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:52:24.587618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:24.587618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:24.587618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:24.587618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:24.587618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:24.587618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:24.587618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:24.587618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:24.587618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:24.587618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:24.587618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:24.587618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:52:24.587618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:52:24.587618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:52:24.589457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:52:24.589457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:24.589457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:24.589457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:24.589457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:24.589457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:24.589457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:24.589457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:24.589457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:52:24.589457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:24.589457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:24.589457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:24.589457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:24.589457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:24.589457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:24.589457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:24.589457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:24.590676 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:52:24.590676 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:52:24.590676 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:52:24.590676 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:52:24.590676 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:52:24.590676 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:52:24.590676 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:52:24.590676 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:52:24.590676 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 619/1244] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:52:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:24.595729 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68ea67-bd99-412c-9a20-2461f725be75 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:24.701083 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68ea67-bd99-412c-9a20-2461f725be75 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:24.736447 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68b921-2dd5-48fe-b13d-4fbad4fe8ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:24.737698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68b921-2dd5-48fe-b13d-4fbad4fe8ac9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:24.738306 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68b921-2dd5-48fe-b13d-4fbad4fe8ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:24.738558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68b921-2dd5-48fe-b13d-4fbad4fe8ac9 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:24.739575 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f68b921-2dd5-48fe-b13d-4fbad4fe8ac9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:24.739937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f68b921-2dd5-48fe-b13d-4fbad4fe8ac9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:24.742300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f68b921-2dd5-48fe-b13d-4fbad4fe8ac9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:24.756992 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 626/1245] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:52:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:24.763766 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927739e-4eee-4072-b7bb-7b4bfbde95bf None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:24.870887 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927739e-4eee-4072-b7bb-7b4bfbde95bf None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:24.907923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050534c-0dca-4c61-beba-5d1c06c962aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:24.908565 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050534c-0dca-4c61-beba-5d1c06c962aa None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:24.908837 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050534c-0dca-4c61-beba-5d1c06c962aa None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:24.909105 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050534c-0dca-4c61-beba-5d1c06c962aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:24.909911 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8050534c-0dca-4c61-beba-5d1c06c962aa None admin] RBAC: Adding query filter params (role.id=44847d50e6cb4bc7938c59b4e2d99833, scope.project.id=0500e378ea374ca692e230bbd8781318, user.id=cf9bc2fa959d4706b9f9e87509829059) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:24.910241 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8050534c-0dca-4c61-beba-5d1c06c962aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:24.911684 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8050534c-0dca-4c61-beba-5d1c06c962aa None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:24.917870 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 620/1246] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:52:24 2022] GET /identity/v3/role_assignments?user.id=cf9bc2fa959d4706b9f9e87509829059&scope.project.id=0500e378ea374ca692e230bbd8781318&role.id=44847d50e6cb4bc7938c59b4e2d99833 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:52:27.302827 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea0451-eb31-476e-af7f-da4f80d8bc0b None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:27.304427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea0451-eb31-476e-af7f-da4f80d8bc0b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:27.304427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea0451-eb31-476e-af7f-da4f80d8bc0b None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:27.304887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 627/1247] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:52:27.310466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a8ebd-7a43-4f04-bc3f-6c78a5091fa8 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:27.311023 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a8ebd-7a43-4f04-bc3f-6c78a5091fa8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:27.311023 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a8ebd-7a43-4f04-bc3f-6c78a5091fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:27.328773 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d81a8ebd-7a43-4f04-bc3f-6c78a5091fa8 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:52:27.329963 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d81a8ebd-7a43-4f04-bc3f-6c78a5091fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:52:27.404183 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 621/1248] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:52:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:52:27.428520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea0451-eb31-476e-af7f-da4f80d8bc0b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:27.532767 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea0451-eb31-476e-af7f-da4f80d8bc0b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:27.566063 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befee0a-f580-4d9c-92bf-5abd36a8089d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:27.566814 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befee0a-f580-4d9c-92bf-5abd36a8089d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:27.567087 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befee0a-f580-4d9c-92bf-5abd36a8089d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:27.567358 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befee0a-f580-4d9c-92bf-5abd36a8089d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:27.577942 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0befee0a-f580-4d9c-92bf-5abd36a8089d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:52:27.586179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-0befee0a-f580-4d9c-92bf-5abd36a8089d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:52:27.586179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:27.586179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:27.586179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:27.586179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:27.586179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:27.586179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:27.586179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:27.586179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:27.586179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:27.586179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:27.586179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:27.586179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:52:27.586179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:52:27.586179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:52:27.586179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:52:27.587309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:52:27.587309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:52:27.587309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:52:27.587309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:52:27.587309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:27.587309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:27.587309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:27.587309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:27.587309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:27.587309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:27.587309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:27.587309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:52:27.587309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:27.587309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:27.587309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:27.587309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:27.588285 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:27.588285 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:27.588285 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:27.588285 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:27.588285 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:52:27.588285 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:52:27.588285 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:52:27.588285 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:52:27.588285 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:52:27.588285 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:52:27.588285 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:52:27.588285 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:52:27.588285 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 628/1249] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:52:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:27.594046 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a8ebd-7a43-4f04-bc3f-6c78a5091fa8 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:27.704822 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a8ebd-7a43-4f04-bc3f-6c78a5091fa8 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:27.742858 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395956e-912f-4be3-ad9d-a1ec243a395b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:27.743549 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395956e-912f-4be3-ad9d-a1ec243a395b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:27.743909 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395956e-912f-4be3-ad9d-a1ec243a395b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:27.744153 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395956e-912f-4be3-ad9d-a1ec243a395b None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:27.744892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6395956e-912f-4be3-ad9d-a1ec243a395b None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:27.745198 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6395956e-912f-4be3-ad9d-a1ec243a395b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:27.746651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6395956e-912f-4be3-ad9d-a1ec243a395b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:27.757343 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 622/1250] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:52:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:27.763011 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befee0a-f580-4d9c-92bf-5abd36a8089d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:27.871418 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befee0a-f580-4d9c-92bf-5abd36a8089d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:27.908674 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8d767-442c-4b60-ab48-21a01e52132f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:27.909561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8d767-442c-4b60-ab48-21a01e52132f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:27.909877 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8d767-442c-4b60-ab48-21a01e52132f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:27.910218 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8d767-442c-4b60-ab48-21a01e52132f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:27.922790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba8d767-442c-4b60-ab48-21a01e52132f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:27.924469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba8d767-442c-4b60-ab48-21a01e52132f None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:27.943844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-1ba8d767-442c-4b60-ab48-21a01e52132f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:27.943844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:27.943844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:27.943844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:27.943844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:27.943844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:27.943844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:27.943844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:27.943844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:27.943844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:27.943844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:27.943844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:27.943844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:52:27.943844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:52:27.943844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:52:27.945185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:52:27.945185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:27.945185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:27.945185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:27.945185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:27.945185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:27.945185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:27.945185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:27.945185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:52:27.945185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:27.945185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:27.945185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:27.945185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:27.945185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:27.945185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:27.945185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:27.945185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:27.946092 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:52:27.946092 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:52:27.946092 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:27.946092 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:52:27.946092 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 629/1251] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:52:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:27.951056 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395956e-912f-4be3-ad9d-a1ec243a395b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:28.067471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395956e-912f-4be3-ad9d-a1ec243a395b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:28.104522 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d002c-865c-40e2-a171-b6817b080d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:28.105176 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d002c-865c-40e2-a171-b6817b080d2e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:28.105458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d002c-865c-40e2-a171-b6817b080d2e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:28.105822 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d002c-865c-40e2-a171-b6817b080d2e None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:28.106603 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d002c-865c-40e2-a171-b6817b080d2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:28.106923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d002c-865c-40e2-a171-b6817b080d2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:28.108847 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d002c-865c-40e2-a171-b6817b080d2e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:28.121142 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 623/1252] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:28.127602 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8d767-442c-4b60-ab48-21a01e52132f None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:28.232310 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8d767-442c-4b60-ab48-21a01e52132f None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:28.268855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef3154-d465-4b20-94b6-e297630c0375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:28.269676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef3154-d465-4b20-94b6-e297630c0375 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:28.269977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef3154-d465-4b20-94b6-e297630c0375 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:28.270299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef3154-d465-4b20-94b6-e297630c0375 None admin] PATH_INFO: `/v3/users/placement` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:28.278602 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef3154-d465-4b20-94b6-e297630c0375 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:28.280407 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef3154-d465-4b20-94b6-e297630c0375 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:28.296349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-edef3154-d465-4b20-94b6-e297630c0375 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 22:52:28.296349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:28.296349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:28.296349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:28.296349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:28.296349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:28.296349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:28.296349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:28.296349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:28.296349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:28.296349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:28.296349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:28.296349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:52:28.296349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:52:28.296349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:52:28.296349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:52:28.297319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:28.297319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:28.297319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:52:28.297319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:52:28.297319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:52:28.297319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:52:28.297319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:28.297319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:28.297319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:28.297319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:28.297319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:28.297319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:52:28.297319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:28.297319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:28.297319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:28.297319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:28.297319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:28.297319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:28.298321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:28.298321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:28.298321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:52:28.298321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:52:28.298321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:52:28.298321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:52:28.298321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:52:28.298321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:52:28.298321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 22:52:28.298321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:52:28.298321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 630/1253] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:52:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:28.303764 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d002c-865c-40e2-a171-b6817b080d2e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:28.410963 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d002c-865c-40e2-a171-b6817b080d2e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:28.447310 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43347752-5427-495a-b770-9cfefc818311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:28.447976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43347752-5427-495a-b770-9cfefc818311 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:28.448256 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43347752-5427-495a-b770-9cfefc818311 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:28.448529 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43347752-5427-495a-b770-9cfefc818311 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:28.449353 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43347752-5427-495a-b770-9cfefc818311 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:28.449660 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43347752-5427-495a-b770-9cfefc818311 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:28.451111 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43347752-5427-495a-b770-9cfefc818311 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:28.468803 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 624/1254] 10.222.0.23 () {56 vars in 1143 bytes} [Mon May 9 22:52:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:28.474508 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef3154-d465-4b20-94b6-e297630c0375 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:28.583441 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef3154-d465-4b20-94b6-e297630c0375 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:28.619051 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ac0d6-3f49-4204-90f2-213236ba286c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:28.619699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ac0d6-3f49-4204-90f2-213236ba286c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:28.619979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ac0d6-3f49-4204-90f2-213236ba286c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:28.620249 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ac0d6-3f49-4204-90f2-213236ba286c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:28.690312 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ac0d6-3f49-4204-90f2-213236ba286c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:28.691688 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ac0d6-3f49-4204-90f2-213236ba286c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:28.799968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 631/1255] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:52:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 22:52:28.806527 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43347752-5427-495a-b770-9cfefc818311 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:28.922962 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43347752-5427-495a-b770-9cfefc818311 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:28.961878 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe6939-95c2-4e3d-8212-25c6cac919e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:28.962823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe6939-95c2-4e3d-8212-25c6cac919e4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:28.963172 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe6939-95c2-4e3d-8212-25c6cac919e4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:28.963481 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe6939-95c2-4e3d-8212-25c6cac919e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:28.975575 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe6939-95c2-4e3d-8212-25c6cac919e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:28.977014 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe6939-95c2-4e3d-8212-25c6cac919e4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:28.995423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-77fe6939-95c2-4e3d-8212-25c6cac919e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:28.995423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:28.995423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:28.995423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:28.995423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:28.995423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:28.995423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:28.995423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:28.995423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:28.995423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:28.995423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:28.995423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:28.995423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:52:28.995423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:52:28.995423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:52:28.996516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:52:28.996516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:28.996516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:28.996516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:28.996516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:28.996516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:28.996516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:28.996516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:28.996516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:52:28.996516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:28.996516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:28.996516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:28.996516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:28.996516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:28.996516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:28.996516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:28.996516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:28.997506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:52:28.997506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:52:28.997506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:28.997506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:52:28.997506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 625/1256] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:52:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:29.003139 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ac0d6-3f49-4204-90f2-213236ba286c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:29.112509 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ac0d6-3f49-4204-90f2-213236ba286c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:29.147967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389a80d-f454-432a-ad5f-f67567a33031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:29.148631 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389a80d-f454-432a-ad5f-f67567a33031 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:29.149006 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389a80d-f454-432a-ad5f-f67567a33031 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:29.149285 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389a80d-f454-432a-ad5f-f67567a33031 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:29.150002 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389a80d-f454-432a-ad5f-f67567a33031 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:29.150349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389a80d-f454-432a-ad5f-f67567a33031 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:29.151904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389a80d-f454-432a-ad5f-f67567a33031 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:29.163950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 632/1257] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:52:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:29.169949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe6939-95c2-4e3d-8212-25c6cac919e4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:29.277489 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe6939-95c2-4e3d-8212-25c6cac919e4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:29.316096 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9134d-7c51-4075-9d11-50b3486cd570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:29.316894 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9134d-7c51-4075-9d11-50b3486cd570 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:29.317173 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9134d-7c51-4075-9d11-50b3486cd570 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:29.317447 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9134d-7c51-4075-9d11-50b3486cd570 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:29.324944 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9134d-7c51-4075-9d11-50b3486cd570 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:29.326817 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9134d-7c51-4075-9d11-50b3486cd570 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:29.342651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-f2b9134d-7c51-4075-9d11-50b3486cd570 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:52:29.342651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:29.342651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:29.342651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:29.342651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:29.342651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:29.342651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:29.342651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:29.342651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:29.342651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:29.342651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:29.342651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:29.342651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:52:29.342651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:52:29.342651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:52:29.343838 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:52:29.343838 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:29.343838 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:29.343838 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:29.343838 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:29.343838 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:29.343838 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:29.343838 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:29.343838 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:52:29.343838 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:29.343838 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:29.343838 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:29.343838 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:29.343838 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:29.343838 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:29.343838 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:29.343838 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:29.345361 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:52:29.345361 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:52:29.345361 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:52:29.345361 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:52:29.345361 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:52:29.345361 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:52:29.345361 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:52:29.345361 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:52:29.345361 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 626/1258] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:52:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:29.350837 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389a80d-f454-432a-ad5f-f67567a33031 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:29.469918 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389a80d-f454-432a-ad5f-f67567a33031 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:29.506657 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c425c01-c400-480b-8006-e3a107675fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:29.507360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c425c01-c400-480b-8006-e3a107675fbc None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:29.507594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c425c01-c400-480b-8006-e3a107675fbc None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:29.507862 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c425c01-c400-480b-8006-e3a107675fbc None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:29.508650 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c425c01-c400-480b-8006-e3a107675fbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:29.508954 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c425c01-c400-480b-8006-e3a107675fbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:29.510449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c425c01-c400-480b-8006-e3a107675fbc None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:29.524128 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 633/1259] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:52:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:29.530440 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9134d-7c51-4075-9d11-50b3486cd570 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:29.633320 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9134d-7c51-4075-9d11-50b3486cd570 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:29.667276 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb9445-15eb-43ae-8b1a-b85d8c1a1b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:29.667935 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb9445-15eb-43ae-8b1a-b85d8c1a1b00 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:29.668210 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb9445-15eb-43ae-8b1a-b85d8c1a1b00 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:29.668477 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb9445-15eb-43ae-8b1a-b85d8c1a1b00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:29.669301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb9445-15eb-43ae-8b1a-b85d8c1a1b00 None admin] RBAC: Adding query filter params (role.id=bda550492f4f4c5c83f52fa89fc18a7d, scope.project.id=0500e378ea374ca692e230bbd8781318, user.id=cf9bc2fa959d4706b9f9e87509829059) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:29.669601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb9445-15eb-43ae-8b1a-b85d8c1a1b00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:29.671594 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb9445-15eb-43ae-8b1a-b85d8c1a1b00 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:29.677640 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 627/1260] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:52:29 2022] GET /identity/v3/role_assignments?user.id=cf9bc2fa959d4706b9f9e87509829059&scope.project.id=0500e378ea374ca692e230bbd8781318&role.id=bda550492f4f4c5c83f52fa89fc18a7d => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:31.852417 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8b803-bc00-4b76-9b8f-eea8b4f821bb None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:31.852968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8b803-bc00-4b76-9b8f-eea8b4f821bb None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:31.852968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8b803-bc00-4b76-9b8f-eea8b4f821bb None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:31.854384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 634/1261] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:52:31.860096 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42ffa6-5c65-49d4-8461-f82dc22eaf65 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:31.860521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42ffa6-5c65-49d4-8461-f82dc22eaf65 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:31.860623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42ffa6-5c65-49d4-8461-f82dc22eaf65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:31.876654 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-8a42ffa6-5c65-49d4-8461-f82dc22eaf65 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:52:31.877743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8a42ffa6-5c65-49d4-8461-f82dc22eaf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:52:31.938780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 628/1262] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:52:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:52:31.970198 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8b803-bc00-4b76-9b8f-eea8b4f821bb None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:32.064114 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8b803-bc00-4b76-9b8f-eea8b4f821bb None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:32.096103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f9872-8351-4c39-98cb-b87d5ff05dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:32.096872 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f9872-8351-4c39-98cb-b87d5ff05dda None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:32.097148 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f9872-8351-4c39-98cb-b87d5ff05dda None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:32.097417 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f9872-8351-4c39-98cb-b87d5ff05dda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:32.106949 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f9872-8351-4c39-98cb-b87d5ff05dda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:52:32.114828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-690f9872-8351-4c39-98cb-b87d5ff05dda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:52:32.114828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:32.114828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:32.114828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:32.114828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:32.114828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:32.114828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:32.114828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:32.114828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:32.114828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:32.114828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:32.114828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:32.114828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:52:32.114828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:52:32.114828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:52:32.114828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:52:32.115952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:52:32.115952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:52:32.115952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:52:32.115952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:52:32.115952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:32.115952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:32.115952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:32.115952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:32.115952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:32.115952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:32.115952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:32.115952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:52:32.115952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:32.115952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:32.115952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:32.115952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:32.117096 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:32.117096 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:32.117096 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:32.117096 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:32.117096 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:52:32.117096 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:52:32.117096 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:52:32.117096 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:52:32.117096 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:52:32.117096 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:52:32.117096 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:52:32.117096 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:52:32.117096 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 635/1263] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:52:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:32.121683 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42ffa6-5c65-49d4-8461-f82dc22eaf65 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:32.213390 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42ffa6-5c65-49d4-8461-f82dc22eaf65 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:32.245257 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91e79a-ffdf-443f-a93a-820306000eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:32.245898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91e79a-ffdf-443f-a93a-820306000eb8 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:32.246203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91e79a-ffdf-443f-a93a-820306000eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:32.246478 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91e79a-ffdf-443f-a93a-820306000eb8 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:32.247205 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91e79a-ffdf-443f-a93a-820306000eb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:32.247504 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91e79a-ffdf-443f-a93a-820306000eb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:32.248899 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91e79a-ffdf-443f-a93a-820306000eb8 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:32.256476 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 629/1264] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:52:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:32.261956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f9872-8351-4c39-98cb-b87d5ff05dda None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:32.355385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f9872-8351-4c39-98cb-b87d5ff05dda None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:32.386605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a927e-a2d7-4545-90a9-9744ca906e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:32.387367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a927e-a2d7-4545-90a9-9744ca906e9e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:32.387642 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a927e-a2d7-4545-90a9-9744ca906e9e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:32.387912 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a927e-a2d7-4545-90a9-9744ca906e9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:32.398416 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a927e-a2d7-4545-90a9-9744ca906e9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:32.399831 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a927e-a2d7-4545-90a9-9744ca906e9e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:32.415788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-761a927e-a2d7-4545-90a9-9744ca906e9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:32.415788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:32.415788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:32.415788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:32.415788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:32.415788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:32.415788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:32.415788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:32.415788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:32.415788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:32.415788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:32.415788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:32.415788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:52:32.415788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:52:32.415788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:52:32.416905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:52:32.416905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:32.416905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:32.416905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:32.416905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:32.416905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:32.416905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:32.416905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:32.416905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:52:32.416905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:32.416905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:32.416905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:32.416905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:32.416905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:32.416905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:32.416905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:32.416905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:32.418007 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:52:32.418007 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:52:32.418007 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:32.418007 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:52:32.418007 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 636/1265] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:52:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:32.422526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91e79a-ffdf-443f-a93a-820306000eb8 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:32.517073 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91e79a-ffdf-443f-a93a-820306000eb8 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:32.547796 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0982d97-e2dc-42c1-af09-96db2cc297cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:32.548433 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0982d97-e2dc-42c1-af09-96db2cc297cd None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:32.548714 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0982d97-e2dc-42c1-af09-96db2cc297cd None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:32.548988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0982d97-e2dc-42c1-af09-96db2cc297cd None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:32.549695 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0982d97-e2dc-42c1-af09-96db2cc297cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:32.549997 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0982d97-e2dc-42c1-af09-96db2cc297cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:32.551372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0982d97-e2dc-42c1-af09-96db2cc297cd None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:32.562902 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 630/1266] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:32.568359 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a927e-a2d7-4545-90a9-9744ca906e9e None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:32.663957 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a927e-a2d7-4545-90a9-9744ca906e9e None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:32.695269 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59bccdd-d406-4802-84f5-afa5dd633665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:32.696045 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bccdd-d406-4802-84f5-afa5dd633665 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:32.696326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bccdd-d406-4802-84f5-afa5dd633665 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:32.696600 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bccdd-d406-4802-84f5-afa5dd633665 None admin] PATH_INFO: `/v3/users/placement` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:32.704267 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59bccdd-d406-4802-84f5-afa5dd633665 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:32.705682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59bccdd-d406-4802-84f5-afa5dd633665 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:32.720033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-c59bccdd-d406-4802-84f5-afa5dd633665 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 22:52:32.720033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:32.720033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:32.720033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:32.720033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:32.720033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:32.720033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:32.720033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:32.720033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:32.720033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:32.720033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:32.720033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:32.720033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:52:32.720033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:52:32.720033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:52:32.720033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:52:32.721175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:32.721175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:32.721175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:52:32.721175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:52:32.721175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:52:32.721175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:52:32.721175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:32.721175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:32.721175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:32.721175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:32.721175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:32.721175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:52:32.721175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:32.721175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:32.721175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:32.721175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:32.721175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:32.721175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:32.722387 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:32.722387 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:32.722387 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:52:32.722387 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:52:32.722387 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:52:32.722387 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:52:32.722387 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:52:32.722387 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:52:32.722387 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 22:52:32.722387 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:52:32.722387 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 637/1267] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:52:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:32.726971 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0982d97-e2dc-42c1-af09-96db2cc297cd None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:32.836364 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0982d97-e2dc-42c1-af09-96db2cc297cd None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:32.868629 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d241514-0a32-4947-909f-fe1cf951d6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:32.869281 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d241514-0a32-4947-909f-fe1cf951d6f8 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:32.869557 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d241514-0a32-4947-909f-fe1cf951d6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:32.869825 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d241514-0a32-4947-909f-fe1cf951d6f8 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:32.870672 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d241514-0a32-4947-909f-fe1cf951d6f8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:32.870976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d241514-0a32-4947-909f-fe1cf951d6f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:32.872387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d241514-0a32-4947-909f-fe1cf951d6f8 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:32.888439 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 631/1268] 10.222.0.23 () {56 vars in 1143 bytes} [Mon May 9 22:52:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:32.894579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59bccdd-d406-4802-84f5-afa5dd633665 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:32.989499 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59bccdd-d406-4802-84f5-afa5dd633665 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:33.020957 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afbda1-7310-40ea-98c1-37d053464683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:33.021726 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afbda1-7310-40ea-98c1-37d053464683 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:33.022003 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afbda1-7310-40ea-98c1-37d053464683 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:33.022301 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afbda1-7310-40ea-98c1-37d053464683 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:33.033507 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7afbda1-7310-40ea-98c1-37d053464683 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:33.034967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7afbda1-7310-40ea-98c1-37d053464683 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:33.050958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-e7afbda1-7310-40ea-98c1-37d053464683 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:33.050958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:33.050958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:33.050958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:33.050958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:33.050958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:33.050958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:33.050958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:33.050958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:33.050958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:33.050958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:33.050958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:33.050958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:52:33.050958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:52:33.050958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:52:33.052023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:52:33.052023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:33.052023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:33.052023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:33.052023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:33.052023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:33.052023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:33.052023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:33.052023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:52:33.052023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:33.052023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:33.052023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:33.052023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:33.052023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:33.052023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:33.052023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:33.052023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:33.053145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:52:33.053145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:52:33.053145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:33.053145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:52:33.053145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 638/1269] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:52:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:33.057863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d241514-0a32-4947-909f-fe1cf951d6f8 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:33.151428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d241514-0a32-4947-909f-fe1cf951d6f8 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:33.183158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fdf344-fcb8-4071-bd68-ad2f9c7067f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:33.183797 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fdf344-fcb8-4071-bd68-ad2f9c7067f4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:33.184077 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fdf344-fcb8-4071-bd68-ad2f9c7067f4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:33.184351 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fdf344-fcb8-4071-bd68-ad2f9c7067f4 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:33.185058 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fdf344-fcb8-4071-bd68-ad2f9c7067f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:33.185359 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fdf344-fcb8-4071-bd68-ad2f9c7067f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:33.187246 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fdf344-fcb8-4071-bd68-ad2f9c7067f4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:33.197837 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 632/1270] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:33.203936 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afbda1-7310-40ea-98c1-37d053464683 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:33.298467 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afbda1-7310-40ea-98c1-37d053464683 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:33.330001 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265429f-f95c-4c52-89ae-9e6139fc04f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:33.330794 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265429f-f95c-4c52-89ae-9e6139fc04f2 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:33.331073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265429f-f95c-4c52-89ae-9e6139fc04f2 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:33.331344 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265429f-f95c-4c52-89ae-9e6139fc04f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:33.337629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f265429f-f95c-4c52-89ae-9e6139fc04f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:33.339076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f265429f-f95c-4c52-89ae-9e6139fc04f2 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:33.352633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-f265429f-f95c-4c52-89ae-9e6139fc04f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:52:33.352633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:33.352633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:33.352633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:33.352633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:33.352633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:33.352633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:33.352633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:33.352633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:33.352633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:33.352633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:33.352633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:33.352633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:52:33.352633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:52:33.352633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:52:33.353656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:52:33.353656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:33.353656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:33.353656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:33.353656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:33.353656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:33.353656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:33.353656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:33.353656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:52:33.353656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:33.353656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:33.353656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:33.353656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:33.353656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:33.353656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:33.353656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:33.353656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:33.354939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:52:33.354939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:52:33.354939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:52:33.354939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:52:33.354939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:52:33.354939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:52:33.354939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:52:33.354939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:52:33.354939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 639/1271] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:52:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:33.359733 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fdf344-fcb8-4071-bd68-ad2f9c7067f4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:33.453507 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fdf344-fcb8-4071-bd68-ad2f9c7067f4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:33.484910 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c69a4-2238-43c5-884e-428ff2a9e1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:33.485548 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c69a4-2238-43c5-884e-428ff2a9e1d3 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:33.485828 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c69a4-2238-43c5-884e-428ff2a9e1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:33.486103 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c69a4-2238-43c5-884e-428ff2a9e1d3 None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:33.486904 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c69a4-2238-43c5-884e-428ff2a9e1d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:33.487210 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c69a4-2238-43c5-884e-428ff2a9e1d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:33.488623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c69a4-2238-43c5-884e-428ff2a9e1d3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:33.500453 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 633/1272] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:52:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:33.506256 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265429f-f95c-4c52-89ae-9e6139fc04f2 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:33.597265 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265429f-f95c-4c52-89ae-9e6139fc04f2 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:33.630221 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe84c6-393a-49f8-b6fb-2022b3693a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:33.631102 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe84c6-393a-49f8-b6fb-2022b3693a4c None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:33.631383 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe84c6-393a-49f8-b6fb-2022b3693a4c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:33.631653 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe84c6-393a-49f8-b6fb-2022b3693a4c None admin] PATH_INFO: `/v3/projects/0500e378ea374ca692e230bbd8781318/users/cf9bc2fa959d4706b9f9e87509829059/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:33.633633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe84c6-393a-49f8-b6fb-2022b3693a4c None admin] RBAC: Authorizing `identity:create_grant(project_id=0500e378ea374ca692e230bbd8781318, user_id=cf9bc2fa959d4706b9f9e87509829059, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:33.635098 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe84c6-393a-49f8-b6fb-2022b3693a4c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:33.647207 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 640/1273] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 9 22:52:33 2022] PUT /identity/v3/projects/0500e378ea374ca692e230bbd8781318/users/cf9bc2fa959d4706b9f9e87509829059/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 22:52:35.800517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49b23f-8acc-4ffa-bff2-f0e26d1c5c36 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:35.801068 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49b23f-8acc-4ffa-bff2-f0e26d1c5c36 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:35.801068 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49b23f-8acc-4ffa-bff2-f0e26d1c5c36 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:35.802439 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 634/1274] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:52:35.807482 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be8621-4a9a-4a08-8818-846112cda8ae None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:35.807904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be8621-4a9a-4a08-8818-846112cda8ae None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:35.808039 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be8621-4a9a-4a08-8818-846112cda8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:35.824856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-96be8621-4a9a-4a08-8818-846112cda8ae None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:52:35.826027 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-96be8621-4a9a-4a08-8818-846112cda8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:52:35.888390 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 641/1275] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:52:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:52:35.910398 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49b23f-8acc-4ffa-bff2-f0e26d1c5c36 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:36.004855 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49b23f-8acc-4ffa-bff2-f0e26d1c5c36 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:36.036168 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbcacda-850a-45e3-9960-1ec8c3475371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:36.036928 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcacda-850a-45e3-9960-1ec8c3475371 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:36.037208 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcacda-850a-45e3-9960-1ec8c3475371 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:36.037479 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcacda-850a-45e3-9960-1ec8c3475371 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:36.047533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbcacda-850a-45e3-9960-1ec8c3475371 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 22:52:36.055036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-9fbcacda-850a-45e3-9960-1ec8c3475371 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:52:36.055036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:36.055036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:36.055036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:36.055036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:36.055036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:36.055036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:36.055036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:36.055036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:36.055036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:36.055036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:36.055036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:36.055036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 22:52:36.055036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 22:52:36.055036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 22:52:36.055036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 22:52:36.056110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 22:52:36.056110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 22:52:36.056110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 22:52:36.056110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 22:52:36.056110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:36.056110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:36.056110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:36.056110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:36.056110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:36.056110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:36.056110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:36.056110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:52:36.056110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:36.056110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:36.056110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:36.056110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:36.057003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:36.057003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:36.057003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:36.057003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:36.057003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 22:52:36.057003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 22:52:36.057003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 22:52:36.057003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 22:52:36.057003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 22:52:36.057003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 22:52:36.057003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 22:52:36.057003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:52:36.057003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 635/1276] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 9 22:52:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:36.062164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be8621-4a9a-4a08-8818-846112cda8ae None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:36.153278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be8621-4a9a-4a08-8818-846112cda8ae None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:36.185498 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89b450-27d1-494e-8f58-8116ffab0806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:36.186194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89b450-27d1-494e-8f58-8116ffab0806 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:36.186479 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89b450-27d1-494e-8f58-8116ffab0806 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:36.186752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89b450-27d1-494e-8f58-8116ffab0806 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:36.187481 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a89b450-27d1-494e-8f58-8116ffab0806 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:36.187785 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a89b450-27d1-494e-8f58-8116ffab0806 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:36.189148 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a89b450-27d1-494e-8f58-8116ffab0806 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:36.196719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 642/1277] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 9 22:52:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 22:52:36.202459 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbcacda-850a-45e3-9960-1ec8c3475371 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:36.296247 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbcacda-850a-45e3-9960-1ec8c3475371 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:36.327173 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbc0ec-d88e-4dc8-8a67-44ceb6d0b6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:36.327942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbc0ec-d88e-4dc8-8a67-44ceb6d0b6e1 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:36.328220 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbc0ec-d88e-4dc8-8a67-44ceb6d0b6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:36.328493 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbc0ec-d88e-4dc8-8a67-44ceb6d0b6e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:36.338491 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecbc0ec-d88e-4dc8-8a67-44ceb6d0b6e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:36.339921 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecbc0ec-d88e-4dc8-8a67-44ceb6d0b6e1 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:36.356534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-4ecbc0ec-d88e-4dc8-8a67-44ceb6d0b6e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:36.356534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:36.356534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:36.356534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:36.356534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:36.356534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:36.356534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:36.356534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:36.356534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:36.356534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:36.356534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:36.356534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:36.356534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:52:36.356534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:52:36.356534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:52:36.357576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:52:36.357576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:36.357576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:36.357576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:36.357576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:36.357576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:36.357576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:36.357576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:36.357576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:52:36.357576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:36.357576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:36.357576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:36.357576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:36.357576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:36.357576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:36.357576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:36.357576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:36.359698 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:52:36.359698 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:52:36.359698 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:36.359698 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:52:36.359698 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 636/1278] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:52:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:36.364145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89b450-27d1-494e-8f58-8116ffab0806 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:36.455758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89b450-27d1-494e-8f58-8116ffab0806 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:36.486871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b385fc-d499-4bdf-9300-ef6b2f25dfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:36.487512 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b385fc-d499-4bdf-9300-ef6b2f25dfcc None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:36.487786 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b385fc-d499-4bdf-9300-ef6b2f25dfcc None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:36.488054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b385fc-d499-4bdf-9300-ef6b2f25dfcc None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:36.488752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b385fc-d499-4bdf-9300-ef6b2f25dfcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:36.489050 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b385fc-d499-4bdf-9300-ef6b2f25dfcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:36.490432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b385fc-d499-4bdf-9300-ef6b2f25dfcc None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:36.501895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 643/1279] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:36.507440 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbc0ec-d88e-4dc8-8a67-44ceb6d0b6e1 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:36.601454 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbc0ec-d88e-4dc8-8a67-44ceb6d0b6e1 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:36.632941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa515dad-5be6-412f-9af1-62afaa50bcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:36.633777 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa515dad-5be6-412f-9af1-62afaa50bcfa None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:36.634058 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa515dad-5be6-412f-9af1-62afaa50bcfa None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:36.634357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa515dad-5be6-412f-9af1-62afaa50bcfa None admin] PATH_INFO: `/v3/users/placement` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:36.642257 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa515dad-5be6-412f-9af1-62afaa50bcfa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:36.643680 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa515dad-5be6-412f-9af1-62afaa50bcfa None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:36.658072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-aa515dad-5be6-412f-9af1-62afaa50bcfa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 22:52:36.658072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:36.658072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:36.658072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:36.658072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:36.658072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:36.658072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:36.658072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:36.658072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:36.658072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:36.658072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:36.658072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:36.658072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 22:52:36.658072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 22:52:36.658072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 22:52:36.658072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 22:52:36.659224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:36.659224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:36.659224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 22:52:36.659224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:52:36.659224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 22:52:36.659224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 22:52:36.659224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:36.659224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:36.659224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:36.659224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:36.659224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:36.659224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:52:36.659224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:36.659224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:36.659224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:36.659224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:36.659224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:36.659224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:36.660270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:36.660270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:36.660270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 22:52:36.660270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 22:52:36.660270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 22:52:36.660270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 22:52:36.660270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 22:52:36.660270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 22:52:36.660270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 22:52:36.660270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:52:36.660270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 637/1280] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:52:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:36.664883 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b385fc-d499-4bdf-9300-ef6b2f25dfcc None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:36.762032 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b385fc-d499-4bdf-9300-ef6b2f25dfcc None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:36.798824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d27fe-73fd-45a5-9dec-ea36e1fd6cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:36.799486 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d27fe-73fd-45a5-9dec-ea36e1fd6cd8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:36.799805 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d27fe-73fd-45a5-9dec-ea36e1fd6cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:36.800059 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d27fe-73fd-45a5-9dec-ea36e1fd6cd8 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:36.800860 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d27fe-73fd-45a5-9dec-ea36e1fd6cd8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:36.801174 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d27fe-73fd-45a5-9dec-ea36e1fd6cd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:36.802627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d27fe-73fd-45a5-9dec-ea36e1fd6cd8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:36.819902 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 644/1281] 10.222.0.23 () {56 vars in 1143 bytes} [Mon May 9 22:52:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:36.825555 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa515dad-5be6-412f-9af1-62afaa50bcfa None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:36.919748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa515dad-5be6-412f-9af1-62afaa50bcfa None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:36.951295 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345db66f-7c17-499d-a997-ccfaed2c0752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:36.952503 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345db66f-7c17-499d-a997-ccfaed2c0752 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:36.952779 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345db66f-7c17-499d-a997-ccfaed2c0752 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:36.953049 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345db66f-7c17-499d-a997-ccfaed2c0752 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:37.016270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345db66f-7c17-499d-a997-ccfaed2c0752 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:37.017641 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345db66f-7c17-499d-a997-ccfaed2c0752 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:37.107073 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 638/1282] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 9 22:52:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 22:52:37.113046 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d27fe-73fd-45a5-9dec-ea36e1fd6cd8 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:37.206488 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d27fe-73fd-45a5-9dec-ea36e1fd6cd8 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:37.240609 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1058e90-f21b-4048-9806-c78dd55054bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:37.241382 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1058e90-f21b-4048-9806-c78dd55054bd None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:37.241662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1058e90-f21b-4048-9806-c78dd55054bd None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:37.241933 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1058e90-f21b-4048-9806-c78dd55054bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:37.252065 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1058e90-f21b-4048-9806-c78dd55054bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:37.253498 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1058e90-f21b-4048-9806-c78dd55054bd None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:37.270583 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-c1058e90-f21b-4048-9806-c78dd55054bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:37.270583 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:37.270583 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:37.270583 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:37.270583 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:37.270583 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:37.270583 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:37.270583 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:37.270583 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:37.270583 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:37.270583 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:37.270583 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:37.270583 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 22:52:37.270583 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 22:52:37.270583 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 22:52:37.271632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 22:52:37.271632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:37.271632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:37.271632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:37.271632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:37.271632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:37.271632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:37.271632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:37.271632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:52:37.271632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:37.271632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:37.271632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:37.271632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:37.271632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:37.271632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:37.271632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:37.271632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:37.272821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 22:52:37.272821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 22:52:37.272821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 22:52:37.272821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:52:37.272821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 645/1283] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:52:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:37.277347 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345db66f-7c17-499d-a997-ccfaed2c0752 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:37.371967 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345db66f-7c17-499d-a997-ccfaed2c0752 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:37.403349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134d597-242f-4178-9dd8-7e4d2b5c8619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:37.403992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134d597-242f-4178-9dd8-7e4d2b5c8619 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:37.404266 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134d597-242f-4178-9dd8-7e4d2b5c8619 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:37.404540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134d597-242f-4178-9dd8-7e4d2b5c8619 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:37.405269 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3134d597-242f-4178-9dd8-7e4d2b5c8619 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:37.405568 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3134d597-242f-4178-9dd8-7e4d2b5c8619 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:37.406937 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3134d597-242f-4178-9dd8-7e4d2b5c8619 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:37.417457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 639/1284] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 9 22:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:37.423624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1058e90-f21b-4048-9806-c78dd55054bd None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:37.516451 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1058e90-f21b-4048-9806-c78dd55054bd None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:37.548702 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0efa6-c26a-49b2-a273-459ea9b6fa16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:37.549476 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0efa6-c26a-49b2-a273-459ea9b6fa16 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:37.549756 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0efa6-c26a-49b2-a273-459ea9b6fa16 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:37.550029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0efa6-c26a-49b2-a273-459ea9b6fa16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:37.556577 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd0efa6-c26a-49b2-a273-459ea9b6fa16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:37.557998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd0efa6-c26a-49b2-a273-459ea9b6fa16 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:37.571321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-bfd0efa6-c26a-49b2-a273-459ea9b6fa16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:52:37.571321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:37.571321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:37.571321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:37.571321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:37.571321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:37.571321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:37.571321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:37.571321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:37.571321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:37.571321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:37.571321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:37.571321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 22:52:37.571321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 22:52:37.571321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 22:52:37.572439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 22:52:37.572439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:37.572439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:37.572439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:37.572439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:37.572439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:37.572439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:37.572439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:37.572439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 09 22:52:37.572439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:37.572439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:37.572439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:37.572439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:37.572439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:37.572439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:37.572439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:37.572439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:37.573589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 22:52:37.573589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 22:52:37.573589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 22:52:37.573589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 22:52:37.573589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 22:52:37.573589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 22:52:37.573589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 22:52:37.573589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 09 22:52:37.573589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 646/1285] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 9 22:52:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:37.578764 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134d597-242f-4178-9dd8-7e4d2b5c8619 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:37.673484 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134d597-242f-4178-9dd8-7e4d2b5c8619 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:37.703862 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d29fd-133c-44ea-a678-87b9601c0308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:37.704506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d29fd-133c-44ea-a678-87b9601c0308 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:37.704785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d29fd-133c-44ea-a678-87b9601c0308 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:37.705057 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d29fd-133c-44ea-a678-87b9601c0308 None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:37.705823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d29fd-133c-44ea-a678-87b9601c0308 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:37.706148 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d29fd-133c-44ea-a678-87b9601c0308 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:37.707562 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d29fd-133c-44ea-a678-87b9601c0308 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:37.719310 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 640/1286] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 9 22:52:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:37.725167 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0efa6-c26a-49b2-a273-459ea9b6fa16 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:37.821628 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0efa6-c26a-49b2-a273-459ea9b6fa16 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:37.856234 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d9c7e-4caf-4cd0-99e5-2319932847ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:37.859527 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d9c7e-4caf-4cd0-99e5-2319932847ad None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:37.859827 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d9c7e-4caf-4cd0-99e5-2319932847ad None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:37.860107 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d9c7e-4caf-4cd0-99e5-2319932847ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:37.860939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d9c7e-4caf-4cd0-99e5-2319932847ad None admin] RBAC: Adding query filter params (role.id=bda550492f4f4c5c83f52fa89fc18a7d, scope.project.id=0500e378ea374ca692e230bbd8781318, user.id=cf9bc2fa959d4706b9f9e87509829059) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:37.861254 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d9c7e-4caf-4cd0-99e5-2319932847ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:37.862730 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d9c7e-4caf-4cd0-99e5-2319932847ad None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:37.868608 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 647/1287] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 9 22:52:37 2022] GET /identity/v3/role_assignments?user.id=cf9bc2fa959d4706b9f9e87509829059&scope.project.id=0500e378ea374ca692e230bbd8781318&role.id=bda550492f4f4c5c83f52fa89fc18a7d => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:40.107476 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b029cb-c92e-4fc0-b725-35ee501996fd None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:40.107999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b029cb-c92e-4fc0-b725-35ee501996fd None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:40.107999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b029cb-c92e-4fc0-b725-35ee501996fd None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:40.109638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 641/1288] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:52:40.114867 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae128bc-90dd-416f-8dbe-1afa323cc8b7 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:40.115282 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae128bc-90dd-416f-8dbe-1afa323cc8b7 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:40.115418 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae128bc-90dd-416f-8dbe-1afa323cc8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:40.132008 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-8ae128bc-90dd-416f-8dbe-1afa323cc8b7 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:52:40.133164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-8ae128bc-90dd-416f-8dbe-1afa323cc8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:52:40.202138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 648/1289] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:52:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:52:40.223835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b029cb-c92e-4fc0-b725-35ee501996fd None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:40.319204 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b029cb-c92e-4fc0-b725-35ee501996fd None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:40.350268 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f645a-4905-4744-a2a5-330892545607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:40.351041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f645a-4905-4744-a2a5-330892545607 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:40.351322 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f645a-4905-4744-a2a5-330892545607 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:40.351596 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f645a-4905-4744-a2a5-330892545607 None admin] PATH_INFO: `/v3/services/placement` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:40.352236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f645a-4905-4744-a2a5-330892545607 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:40.353594 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f645a-4905-4744-a2a5-330892545607 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:40.365853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-8f5f645a-4905-4744-a2a5-330892545607 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 09 22:52:40.365853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:40.365853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:52:40.365853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:52:40.365853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:52:40.365853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:52:40.365853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:52:40.365853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:52:40.365853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 22:52:40.365853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:52:40.365853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:52:40.365853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:52:40.365853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:40.365853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:40.365853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:40.365853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:40.365853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:40.367086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:40.367086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:40.367086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:40.367086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:40.367086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:40.367086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:40.367086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:52:40.367086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:52:40.367086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:52:40.367086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:52:40.367086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:40.367086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:40.367086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:40.367086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:40.367086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:40.367086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:40.368109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:40.368109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:52:40.368109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:40.368109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:40.368109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:40.368109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:40.368109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:40.368109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:40.368109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:40.368109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:40.368109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:52:40.368109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:52:40.368109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 22:52:40.368109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:52:40.368109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 642/1290] 10.222.0.23 () {56 vars in 1117 bytes} [Mon May 9 22:52:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:40.372817 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae128bc-90dd-416f-8dbe-1afa323cc8b7 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:40.467404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae128bc-90dd-416f-8dbe-1afa323cc8b7 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:40.504777 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb0194-4719-4389-9425-f7b0c18456a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:40.505428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb0194-4719-4389-9425-f7b0c18456a7 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:40.505706 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb0194-4719-4389-9425-f7b0c18456a7 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:40.505978 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb0194-4719-4389-9425-f7b0c18456a7 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:40.506703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb0194-4719-4389-9425-f7b0c18456a7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:40.507006 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb0194-4719-4389-9425-f7b0c18456a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:40.508486 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb0194-4719-4389-9425-f7b0c18456a7 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:40.513213 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 649/1291] 10.222.0.23 () {56 vars in 1116 bytes} [Mon May 9 22:52:40 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:40.518875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f645a-4905-4744-a2a5-330892545607 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:40.613832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f645a-4905-4744-a2a5-330892545607 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:40.645218 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd55dc0d-f39d-4810-955c-9bff53363dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:40.645872 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55dc0d-f39d-4810-955c-9bff53363dce None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:40.646178 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55dc0d-f39d-4810-955c-9bff53363dce None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:40.646455 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55dc0d-f39d-4810-955c-9bff53363dce None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:40.647149 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd55dc0d-f39d-4810-955c-9bff53363dce None admin] RBAC: Adding query filter params (type=placement) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:40.647449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd55dc0d-f39d-4810-955c-9bff53363dce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:40.648794 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd55dc0d-f39d-4810-955c-9bff53363dce None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:40.653074 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 643/1292] 10.222.0.23 () {56 vars in 1116 bytes} [Mon May 9 22:52:40 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:42.860263 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808f603-ebe9-4123-8b7e-d1389a5f63d8 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:42.860872 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808f603-ebe9-4123-8b7e-d1389a5f63d8 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:42.860872 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808f603-ebe9-4123-8b7e-d1389a5f63d8 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:42.862331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 650/1293] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:52:42.870443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dd3a0-cce4-42f1-9e83-ce133f0fd321 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:42.870925 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dd3a0-cce4-42f1-9e83-ce133f0fd321 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:42.871097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dd3a0-cce4-42f1-9e83-ce133f0fd321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:42.890251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-639dd3a0-cce4-42f1-9e83-ce133f0fd321 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:52:42.891520 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-639dd3a0-cce4-42f1-9e83-ce133f0fd321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:52:42.957726 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 644/1294] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:52:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:52:42.980043 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2808f603-ebe9-4123-8b7e-d1389a5f63d8 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:43.074865 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2808f603-ebe9-4123-8b7e-d1389a5f63d8 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:43.106189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406be5e4-a939-465d-8c89-a21c4dfcf339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:43.106854 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406be5e4-a939-465d-8c89-a21c4dfcf339 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:43.107136 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406be5e4-a939-465d-8c89-a21c4dfcf339 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:43.107410 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406be5e4-a939-465d-8c89-a21c4dfcf339 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:43.108937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406be5e4-a939-465d-8c89-a21c4dfcf339 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:43.110309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406be5e4-a939-465d-8c89-a21c4dfcf339 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:43.122992 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 651/1295] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 9 22:52:42 2022] POST /identity/v3/services => generated 246 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:52:45.324136 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f976452-dff5-4cd7-a851-66c6b821da7f None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:45.324666 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f976452-dff5-4cd7-a851-66c6b821da7f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:45.324666 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f976452-dff5-4cd7-a851-66c6b821da7f None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:45.326041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 645/1296] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:52:45.331435 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2307a5f-88e0-459a-9949-2417358b9b9c None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:45.331925 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2307a5f-88e0-459a-9949-2417358b9b9c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:45.332401 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2307a5f-88e0-459a-9949-2417358b9b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:45.349186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f2307a5f-88e0-459a-9949-2417358b9b9c None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:52:45.350382 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f2307a5f-88e0-459a-9949-2417358b9b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:52:45.418093 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 652/1297] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:52:45 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:52:45.440550 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f976452-dff5-4cd7-a851-66c6b821da7f None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:45.540051 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f976452-dff5-4cd7-a851-66c6b821da7f None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:45.571003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac7745-e0bc-4fcc-9ed0-73ef02ffe2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:45.571778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac7745-e0bc-4fcc-9ed0-73ef02ffe2a7 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:45.572055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac7745-e0bc-4fcc-9ed0-73ef02ffe2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:45.572325 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac7745-e0bc-4fcc-9ed0-73ef02ffe2a7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:45.572963 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac7745-e0bc-4fcc-9ed0-73ef02ffe2a7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:45.574540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac7745-e0bc-4fcc-9ed0-73ef02ffe2a7 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:45.586513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-42ac7745-e0bc-4fcc-9ed0-73ef02ffe2a7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 09 22:52:45.586513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:45.586513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:52:45.586513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:52:45.586513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:52:45.586513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:52:45.586513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:52:45.586513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:52:45.586513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 22:52:45.586513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:52:45.586513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:52:45.586513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:52:45.586513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:45.586513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:45.586513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:45.586513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:45.586513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:45.587653 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:45.587653 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:45.587653 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:45.587653 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:45.587653 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:45.587653 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:45.587653 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:52:45.587653 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:52:45.587653 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:52:45.587653 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:52:45.587653 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:45.587653 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:45.587653 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:45.587653 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:45.587653 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:45.587653 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:45.588521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:45.588521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:52:45.588521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:45.588521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:45.588521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:45.588521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:45.588521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:45.588521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:45.588521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:45.588521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:45.588521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:52:45.588521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:52:45.588521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 22:52:45.588521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:52:45.588521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 646/1298] 10.222.0.23 () {56 vars in 1117 bytes} [Mon May 9 22:52:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:45.594398 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2307a5f-88e0-459a-9949-2417358b9b9c None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:45.687583 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2307a5f-88e0-459a-9949-2417358b9b9c None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:45.719515 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f24ce-66ee-4dc3-8815-873857ebf94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:45.720161 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f24ce-66ee-4dc3-8815-873857ebf94c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:45.720442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f24ce-66ee-4dc3-8815-873857ebf94c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:45.720714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f24ce-66ee-4dc3-8815-873857ebf94c None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:45.721402 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f24ce-66ee-4dc3-8815-873857ebf94c None admin] RBAC: Adding query filter params (name=placement) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:45.721703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f24ce-66ee-4dc3-8815-873857ebf94c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:45.723094 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f24ce-66ee-4dc3-8815-873857ebf94c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:45.727931 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 653/1299] 10.222.0.23 () {56 vars in 1116 bytes} [Mon May 9 22:52:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:45.734063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac7745-e0bc-4fcc-9ed0-73ef02ffe2a7 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:45.828989 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac7745-e0bc-4fcc-9ed0-73ef02ffe2a7 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:45.859288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d5334-ea43-4946-8fe1-3b100bdeacb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:45.859912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d5334-ea43-4946-8fe1-3b100bdeacb4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:45.860185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d5334-ea43-4946-8fe1-3b100bdeacb4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:45.860452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d5334-ea43-4946-8fe1-3b100bdeacb4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:45.861213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d5334-ea43-4946-8fe1-3b100bdeacb4 None admin] RBAC: Adding query filter params (interface=public, service_id=f84d4d3ca9d6403296198d983fbd5856, region_id=RegionOne) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:45.861521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d5334-ea43-4946-8fe1-3b100bdeacb4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:45.862880 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d5334-ea43-4946-8fe1-3b100bdeacb4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:45.867551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 647/1300] 10.222.0.23 () {56 vars in 1251 bytes} [Mon May 9 22:52:45 2022] GET /identity/v3/endpoints?service_id=f84d4d3ca9d6403296198d983fbd5856&interface=public®ion_id=RegionOne => generated 194 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:45.873520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f24ce-66ee-4dc3-8815-873857ebf94c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:45.964816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f24ce-66ee-4dc3-8815-873857ebf94c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:45.995915 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae14ca-0b26-4eaa-8c55-fe279355686a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:45.996552 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae14ca-0b26-4eaa-8c55-fe279355686a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:45.996831 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae14ca-0b26-4eaa-8c55-fe279355686a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:45.997105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae14ca-0b26-4eaa-8c55-fe279355686a None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:45.997770 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae14ca-0b26-4eaa-8c55-fe279355686a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:45.999687 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae14ca-0b26-4eaa-8c55-fe279355686a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:46.006153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 654/1301] 10.222.0.23 () {56 vars in 1087 bytes} [Mon May 9 22:52:45 2022] GET /identity/v3/services => generated 2191 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 22:52:48.162137 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c3ec4-36eb-414c-99c1-defc7001d562 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:48.162668 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c3ec4-36eb-414c-99c1-defc7001d562 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:48.162760 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c3ec4-36eb-414c-99c1-defc7001d562 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:48.164047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 648/1302] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:52:48.169111 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8602c9-ea3b-463b-a13f-d93f58421c77 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:48.169501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8602c9-ea3b-463b-a13f-d93f58421c77 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:48.169650 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8602c9-ea3b-463b-a13f-d93f58421c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:48.186251 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-8a8602c9-ea3b-463b-a13f-d93f58421c77 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:52:48.187401 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-8a8602c9-ea3b-463b-a13f-d93f58421c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:52:48.248818 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 655/1303] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:52:48 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 22:52:48.270738 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c3ec4-36eb-414c-99c1-defc7001d562 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:48.364517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c3ec4-36eb-414c-99c1-defc7001d562 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:48.394609 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23272ea0-e45b-4926-8a2e-1a7fd7f5eb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:48.395368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23272ea0-e45b-4926-8a2e-1a7fd7f5eb5d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:48.395641 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23272ea0-e45b-4926-8a2e-1a7fd7f5eb5d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:48.395907 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23272ea0-e45b-4926-8a2e-1a7fd7f5eb5d None admin] PATH_INFO: `/v3/services/placement` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:48.396538 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23272ea0-e45b-4926-8a2e-1a7fd7f5eb5d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:48.397867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23272ea0-e45b-4926-8a2e-1a7fd7f5eb5d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:48.409611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-23272ea0-e45b-4926-8a2e-1a7fd7f5eb5d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 09 22:52:48.409611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:48.409611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 22:52:48.409611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 22:52:48.409611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 22:52:48.409611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 22:52:48.409611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 22:52:48.409611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:52:48.409611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 22:52:48.409611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:52:48.409611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 22:52:48.409611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:52:48.409611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 22:52:48.409611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 22:52:48.409611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 22:52:48.409611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 22:52:48.409611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 22:52:48.410810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 22:52:48.410810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 22:52:48.410810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 22:52:48.410810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 22:52:48.410810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 22:52:48.410810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 22:52:48.410810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 22:52:48.410810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 22:52:48.410810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 22:52:48.410810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 22:52:48.410810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 22:52:48.410810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 22:52:48.410810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 22:52:48.410810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 22:52:48.410810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 22:52:48.410810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 09 22:52:48.411710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 22:52:48.411710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 09 22:52:48.411710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 22:52:48.411710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 09 22:52:48.411710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 22:52:48.411710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 22:52:48.411710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 22:52:48.411710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 09 22:52:48.411710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 22:52:48.411710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 09 22:52:48.411710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 22:52:48.411710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 22:52:48.411710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 22:52:48.411710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 09 22:52:48.411710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 649/1304] 10.222.0.23 () {56 vars in 1117 bytes} [Mon May 9 22:52:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 141 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 22:52:48.416582 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8602c9-ea3b-463b-a13f-d93f58421c77 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:48.507257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8602c9-ea3b-463b-a13f-d93f58421c77 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:48.538886 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d6f0f-a96d-487b-adb0-fba536c6616c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:48.539522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d6f0f-a96d-487b-adb0-fba536c6616c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:48.539795 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d6f0f-a96d-487b-adb0-fba536c6616c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:48.540060 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d6f0f-a96d-487b-adb0-fba536c6616c None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:48.540744 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d6f0f-a96d-487b-adb0-fba536c6616c None admin] RBAC: Adding query filter params (name=placement) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:52:48.541040 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d6f0f-a96d-487b-adb0-fba536c6616c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:48.542411 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d6f0f-a96d-487b-adb0-fba536c6616c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:48.547410 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 656/1305] 10.222.0.23 () {56 vars in 1116 bytes} [Mon May 9 22:52:48 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:52:48.553158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23272ea0-e45b-4926-8a2e-1a7fd7f5eb5d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:52:48.647346 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23272ea0-e45b-4926-8a2e-1a7fd7f5eb5d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:52:48.678437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803fb223-e91b-4004-b37e-cfafd5cf194e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:52:48.679016 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803fb223-e91b-4004-b37e-cfafd5cf194e None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:52:48.679290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803fb223-e91b-4004-b37e-cfafd5cf194e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:52:48.679557 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803fb223-e91b-4004-b37e-cfafd5cf194e None admin] PATH_INFO: `/v3/endpoints` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:52:48.680282 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803fb223-e91b-4004-b37e-cfafd5cf194e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:52:48.681630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803fb223-e91b-4004-b37e-cfafd5cf194e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:52:48.698328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 650/1306] 10.222.0.23 () {60 vars in 1144 bytes} [Mon May 9 22:52:48 2022] POST /identity/v3/endpoints => generated 331 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:53:52.434898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0597e-d190-4927-8738-ee1def7e09b3 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:53:52.435940 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0597e-d190-4927-8738-ee1def7e09b3 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:53:52.436422 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0597e-d190-4927-8738-ee1def7e09b3 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:53:52.441822 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 657/1307] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:53:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:53:52.443242 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c466f-366a-49b7-9a89-ca0a3adb57a6 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:53:52.443668 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c466f-366a-49b7-9a89-ca0a3adb57a6 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:53:52.443879 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c466f-366a-49b7-9a89-ca0a3adb57a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:53:52.517578 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-718c466f-366a-49b7-9a89-ca0a3adb57a6 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:53:52.519091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-718c466f-366a-49b7-9a89-ca0a3adb57a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:53:52.638992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 651/1308] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 22:53:52.692627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b570946-1d57-45e9-a309-d54526de447e None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:53:52.692974 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b570946-1d57-45e9-a309-d54526de447e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:53:52.693290 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b570946-1d57-45e9-a309-d54526de447e None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:53:52.695225 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 658/1309] 10.222.0.23 () {54 vars in 964 bytes} [Mon May 9 22:53:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:53:52.701719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10927d9a-2c8f-41a4-83e2-ce68e8919663 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:53:52.702093 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10927d9a-2c8f-41a4-83e2-ce68e8919663 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:53:52.702484 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10927d9a-2c8f-41a4-83e2-ce68e8919663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:53:52.840589 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-10927d9a-2c8f-41a4-83e2-ce68e8919663 None None] MFA Rules not processed for user `96fd3f6dd9e74ba7a4ad159e43e8ae96`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:53:52.842308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-10927d9a-2c8f-41a4-83e2-ce68e8919663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:53:52.940566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 652/1310] 10.222.0.23 () {58 vars in 1061 bytes} [Mon May 9 22:53:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:53:52.950623 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b570946-1d57-45e9-a309-d54526de447e None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:53:52.952389 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9b570946-1d57-45e9-a309-d54526de447e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:53:53.067042 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b570946-1d57-45e9-a309-d54526de447e None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:53:53.143524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0eab6-7d6a-4232-8807-e1911d9b311a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:53:53.144179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0eab6-7d6a-4232-8807-e1911d9b311a service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:53:53.144611 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0eab6-7d6a-4232-8807-e1911d9b311a service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:53:53.144897 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0eab6-7d6a-4232-8807-e1911d9b311a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:53:53.146624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e2a0eab6-7d6a-4232-8807-e1911d9b311a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:53:53.257546 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a0eab6-7d6a-4232-8807-e1911d9b311a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:53:53.259601 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a0eab6-7d6a-4232-8807-e1911d9b311a service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:53:53.279753 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 659/1311] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:53:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:53:55.802816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ef4c1-df1a-459f-900e-bb4fc47bdc88 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:53:55.803323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ef4c1-df1a-459f-900e-bb4fc47bdc88 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:53:55.803431 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ef4c1-df1a-459f-900e-bb4fc47bdc88 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:53:55.804749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 653/1312] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:53:55.811006 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db043cb6-b8ef-470d-bf3f-18d559b8ff4a None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:53:55.811798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db043cb6-b8ef-470d-bf3f-18d559b8ff4a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:53:55.812083 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db043cb6-b8ef-470d-bf3f-18d559b8ff4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:53:55.899345 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-db043cb6-b8ef-470d-bf3f-18d559b8ff4a None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:53:55.900578 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-db043cb6-b8ef-470d-bf3f-18d559b8ff4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:53:55.997878 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 660/1313] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:53:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:53:56.034653 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ef4c1-df1a-459f-900e-bb4fc47bdc88 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:53:56.036179 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c70ef4c1-df1a-459f-900e-bb4fc47bdc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:53:56.138471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ef4c1-df1a-459f-900e-bb4fc47bdc88 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:53:56.214323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c8ad4-71dc-4604-a88e-8fc334350668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:53:56.215879 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c8ad4-71dc-4604-a88e-8fc334350668 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:53:56.216314 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c8ad4-71dc-4604-a88e-8fc334350668 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:53:56.216619 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c8ad4-71dc-4604-a88e-8fc334350668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:53:56.218242 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-113c8ad4-71dc-4604-a88e-8fc334350668 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:53:56.331963 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113c8ad4-71dc-4604-a88e-8fc334350668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:53:56.333881 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113c8ad4-71dc-4604-a88e-8fc334350668 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:53:56.352879 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 654/1314] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:53:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 22:53:58.933700 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a1bdb-94cf-4c3a-bdb8-1d713391e0aa None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:53:58.934252 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a1bdb-94cf-4c3a-bdb8-1d713391e0aa None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:53:58.934346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a1bdb-94cf-4c3a-bdb8-1d713391e0aa None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:53:58.935644 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 661/1315] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:53:58.940836 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426713d-75eb-4180-b22a-3583fe5fcbd8 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:53:58.941258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426713d-75eb-4180-b22a-3583fe5fcbd8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:53:58.941377 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426713d-75eb-4180-b22a-3583fe5fcbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:53:59.002584 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e426713d-75eb-4180-b22a-3583fe5fcbd8 None None] MFA Rules not processed for user `11ec991028e64f3db9ef62861c931821`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:53:59.003752 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e426713d-75eb-4180-b22a-3583fe5fcbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:53:59.095958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 655/1316] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:53:58 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:53:59.148536 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3fc19-d5e6-4481-b5b1-fa20c3d7349a None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:53:59.148747 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3fc19-d5e6-4481-b5b1-fa20c3d7349a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:53:59.149019 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3fc19-d5e6-4481-b5b1-fa20c3d7349a None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:53:59.150418 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 662/1317] 10.222.0.23 () {54 vars in 964 bytes} [Mon May 9 22:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:53:59.156729 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9eb86f-9008-4778-9e1b-c359d56b06c7 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:53:59.157015 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9eb86f-9008-4778-9e1b-c359d56b06c7 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:53:59.157289 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9eb86f-9008-4778-9e1b-c359d56b06c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:53:59.228693 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-da9eb86f-9008-4778-9e1b-c359d56b06c7 None None] MFA Rules not processed for user `96fd3f6dd9e74ba7a4ad159e43e8ae96`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:53:59.229901 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-da9eb86f-9008-4778-9e1b-c359d56b06c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:53:59.303287 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 656/1318] 10.222.0.23 () {58 vars in 1061 bytes} [Mon May 9 22:53:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:53:59.312794 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3fc19-d5e6-4481-b5b1-fa20c3d7349a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:53:59.314347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-88b3fc19-d5e6-4481-b5b1-fa20c3d7349a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:53:59.412701 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3fc19-d5e6-4481-b5b1-fa20c3d7349a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:53:59.481559 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ecb0f7-c1af-4615-86ec-c1b4267d4d2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:53:59.482231 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecb0f7-c1af-4615-86ec-c1b4267d4d2a service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:53:59.482515 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecb0f7-c1af-4615-86ec-c1b4267d4d2a service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:53:59.482788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecb0f7-c1af-4615-86ec-c1b4267d4d2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:53:59.484661 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-19ecb0f7-c1af-4615-86ec-c1b4267d4d2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:53:59.587195 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ecb0f7-c1af-4615-86ec-c1b4267d4d2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:53:59.588622 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ecb0f7-c1af-4615-86ec-c1b4267d4d2a service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:53:59.607601 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 663/1319] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:53:59 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 22:54:02.779945 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a946d29-178f-4c55-b550-bb9409a1a2d6 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:02.780537 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a946d29-178f-4c55-b550-bb9409a1a2d6 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:02.780537 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a946d29-178f-4c55-b550-bb9409a1a2d6 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:02.781845 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 657/1320] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:54:02.787042 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b02ed7-870b-41af-8804-d3c448bba3fe None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:02.787459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b02ed7-870b-41af-8804-d3c448bba3fe None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:02.787588 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b02ed7-870b-41af-8804-d3c448bba3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:02.849822 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c7b02ed7-870b-41af-8804-d3c448bba3fe None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:54:02.851045 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c7b02ed7-870b-41af-8804-d3c448bba3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:02.940676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 664/1321] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:54:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:54:02.979220 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a946d29-178f-4c55-b550-bb9409a1a2d6 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:54:02.980758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7a946d29-178f-4c55-b550-bb9409a1a2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:03.079492 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a946d29-178f-4c55-b550-bb9409a1a2d6 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:54:03.148523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b48c3b-0492-4e4e-86c8-ea1e126685e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:54:03.149153 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b48c3b-0492-4e4e-86c8-ea1e126685e8 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:03.149427 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b48c3b-0492-4e4e-86c8-ea1e126685e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:03.149693 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b48c3b-0492-4e4e-86c8-ea1e126685e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:03.151316 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d4b48c3b-0492-4e4e-86c8-ea1e126685e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:03.253474 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b48c3b-0492-4e4e-86c8-ea1e126685e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:54:03.254857 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b48c3b-0492-4e4e-86c8-ea1e126685e8 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:54:03.272612 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 658/1322] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:54:02 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:54:06.135248 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2b9cb-cf16-4f88-8bc8-4585882190dc None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:06.135832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2b9cb-cf16-4f88-8bc8-4585882190dc None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:06.135832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2b9cb-cf16-4f88-8bc8-4585882190dc None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:06.137165 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 665/1323] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:54:06.142420 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7e73a-1440-4c2d-8b2e-c18142684d31 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:06.142851 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7e73a-1440-4c2d-8b2e-c18142684d31 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:06.142985 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7e73a-1440-4c2d-8b2e-c18142684d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:06.205282 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b6a7e73a-1440-4c2d-8b2e-c18142684d31 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:54:06.206539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b6a7e73a-1440-4c2d-8b2e-c18142684d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:06.300538 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 659/1324] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:54:06.336105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2b9cb-cf16-4f88-8bc8-4585882190dc None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:54:06.337621 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-00f2b9cb-cf16-4f88-8bc8-4585882190dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:06.438045 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2b9cb-cf16-4f88-8bc8-4585882190dc None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:54:06.507449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf2b84-62f7-463d-885f-83d697d386c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:54:06.508088 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf2b84-62f7-463d-885f-83d697d386c0 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:06.508362 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf2b84-62f7-463d-885f-83d697d386c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:06.508629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf2b84-62f7-463d-885f-83d697d386c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:06.510258 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-bcaf2b84-62f7-463d-885f-83d697d386c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:06.614715 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaf2b84-62f7-463d-885f-83d697d386c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:54:06.616139 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaf2b84-62f7-463d-885f-83d697d386c0 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:54:06.633599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 666/1325] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:54:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:54:08.917851 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f66c3d-a34b-4b8a-a75a-e80e9a34d942 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:08.918430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f66c3d-a34b-4b8a-a75a-e80e9a34d942 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:08.918430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f66c3d-a34b-4b8a-a75a-e80e9a34d942 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:08.919713 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 660/1326] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:54:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:54:08.924924 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75644a3-9bb4-473f-b115-08c7d188d13b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:08.925372 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75644a3-9bb4-473f-b115-08c7d188d13b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:08.925473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75644a3-9bb4-473f-b115-08c7d188d13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:08.989031 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e75644a3-9bb4-473f-b115-08c7d188d13b None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:54:08.990289 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e75644a3-9bb4-473f-b115-08c7d188d13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:09.083745 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 667/1327] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:54:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:54:09.119974 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f66c3d-a34b-4b8a-a75a-e80e9a34d942 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:54:09.121623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d4f66c3d-a34b-4b8a-a75a-e80e9a34d942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:09.223537 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f66c3d-a34b-4b8a-a75a-e80e9a34d942 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:54:09.294674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef1339-6abb-40c5-b390-6099a282eafa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:54:09.295347 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef1339-6abb-40c5-b390-6099a282eafa service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:09.295626 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef1339-6abb-40c5-b390-6099a282eafa service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:09.295900 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef1339-6abb-40c5-b390-6099a282eafa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:09.297525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-daef1339-6abb-40c5-b390-6099a282eafa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:09.402068 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef1339-6abb-40c5-b390-6099a282eafa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:54:09.403517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef1339-6abb-40c5-b390-6099a282eafa service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:54:09.421276 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 661/1328] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:54:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:54:11.669374 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dd088-4720-4248-9a0f-e210ecd8b2d1 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:11.669896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dd088-4720-4248-9a0f-e210ecd8b2d1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:11.669990 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dd088-4720-4248-9a0f-e210ecd8b2d1 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:11.671310 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 668/1329] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:54:11.676527 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ccf05-eb60-45a5-a8fe-c1268185cd89 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:11.676948 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ccf05-eb60-45a5-a8fe-c1268185cd89 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:11.677239 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ccf05-eb60-45a5-a8fe-c1268185cd89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:11.744389 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-382ccf05-eb60-45a5-a8fe-c1268185cd89 None None] MFA Rules not processed for user `11ec991028e64f3db9ef62861c931821`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:54:11.745605 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-382ccf05-eb60-45a5-a8fe-c1268185cd89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:11.838471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 662/1330] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:54:11 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:54:11.875904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dd088-4720-4248-9a0f-e210ecd8b2d1 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:54:11.877575 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ec3dd088-4720-4248-9a0f-e210ecd8b2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:11.982057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dd088-4720-4248-9a0f-e210ecd8b2d1 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:54:12.056180 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c48d17-d556-4978-86de-f03b500acc56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:54:12.056843 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c48d17-d556-4978-86de-f03b500acc56 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:12.057124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c48d17-d556-4978-86de-f03b500acc56 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:12.057398 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c48d17-d556-4978-86de-f03b500acc56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:12.059140 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-65c48d17-d556-4978-86de-f03b500acc56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:12.164374 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c48d17-d556-4978-86de-f03b500acc56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:54:12.165737 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c48d17-d556-4978-86de-f03b500acc56 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:54:12.183660 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 669/1331] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:54:11 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:54:14.693278 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f8019-a26b-46ea-9375-55540420d399 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:14.693725 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f8019-a26b-46ea-9375-55540420d399 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:14.693965 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f8019-a26b-46ea-9375-55540420d399 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:14.695452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 663/1332] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:54:14.701168 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164167a-3726-4bf9-8afe-ba32add1b413 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:14.701807 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164167a-3726-4bf9-8afe-ba32add1b413 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:14.702090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164167a-3726-4bf9-8afe-ba32add1b413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:14.771494 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3164167a-3726-4bf9-8afe-ba32add1b413 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:54:14.772692 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3164167a-3726-4bf9-8afe-ba32add1b413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:14.865463 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 670/1333] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:54:14.901129 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f8019-a26b-46ea-9375-55540420d399 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:54:14.903278 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5a3f8019-a26b-46ea-9375-55540420d399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:15.001809 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f8019-a26b-46ea-9375-55540420d399 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:54:15.074209 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd85fb-0d5b-40aa-b997-0a08042d1123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:54:15.074860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd85fb-0d5b-40aa-b997-0a08042d1123 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:15.075140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd85fb-0d5b-40aa-b997-0a08042d1123 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:15.075414 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd85fb-0d5b-40aa-b997-0a08042d1123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:15.077042 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-83fd85fb-0d5b-40aa-b997-0a08042d1123 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:15.181208 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd85fb-0d5b-40aa-b997-0a08042d1123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:54:15.183219 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd85fb-0d5b-40aa-b997-0a08042d1123 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:54:15.200887 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 664/1334] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:54:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:54:17.471944 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6a7fa-924d-4943-8863-c9b3760c774f None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:17.472391 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6a7fa-924d-4943-8863-c9b3760c774f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:17.472506 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6a7fa-924d-4943-8863-c9b3760c774f None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:17.473853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 671/1335] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:54:17.479171 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80910a3f-f760-4458-b203-9a4a82b03126 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:17.479557 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80910a3f-f760-4458-b203-9a4a82b03126 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:17.479716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80910a3f-f760-4458-b203-9a4a82b03126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:17.541256 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-80910a3f-f760-4458-b203-9a4a82b03126 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:54:17.542444 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-80910a3f-f760-4458-b203-9a4a82b03126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:17.632252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 665/1336] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:54:17.671452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f6a7fa-924d-4943-8863-c9b3760c774f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:54:17.672969 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-45f6a7fa-924d-4943-8863-c9b3760c774f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:17.772798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f6a7fa-924d-4943-8863-c9b3760c774f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:54:17.841318 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4466a-9ac5-41fb-933f-cdc850e6b943 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:54:17.841955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4466a-9ac5-41fb-933f-cdc850e6b943 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:17.842273 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4466a-9ac5-41fb-933f-cdc850e6b943 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:17.842547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4466a-9ac5-41fb-933f-cdc850e6b943 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:17.844179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e9f4466a-9ac5-41fb-933f-cdc850e6b943 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:17.945564 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f4466a-9ac5-41fb-933f-cdc850e6b943 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:54:17.946994 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f4466a-9ac5-41fb-933f-cdc850e6b943 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:54:17.963765 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 672/1337] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:54:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:54:20.620715 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcf983-e562-4658-b33a-47b95692e321 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:20.621219 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcf983-e562-4658-b33a-47b95692e321 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:20.621219 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcf983-e562-4658-b33a-47b95692e321 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:20.622661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 666/1338] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:54:20.627877 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95617e1-a10c-43da-9d64-d12fb7356594 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:20.628265 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95617e1-a10c-43da-9d64-d12fb7356594 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:20.628466 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95617e1-a10c-43da-9d64-d12fb7356594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:20.691008 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c95617e1-a10c-43da-9d64-d12fb7356594 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:54:20.692191 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c95617e1-a10c-43da-9d64-d12fb7356594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:20.784028 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 673/1339] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:54:20 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:54:20.820474 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcf983-e562-4658-b33a-47b95692e321 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:54:20.822031 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2bfcf983-e562-4658-b33a-47b95692e321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:20.921323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcf983-e562-4658-b33a-47b95692e321 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:54:20.992361 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23a1bc-9763-4a1e-9b08-533c3e91aa2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:54:20.992998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23a1bc-9763-4a1e-9b08-533c3e91aa2d service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:20.993283 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23a1bc-9763-4a1e-9b08-533c3e91aa2d service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:20.993558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23a1bc-9763-4a1e-9b08-533c3e91aa2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:20.995790 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-aa23a1bc-9763-4a1e-9b08-533c3e91aa2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:21.101199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa23a1bc-9763-4a1e-9b08-533c3e91aa2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:54:21.102658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa23a1bc-9763-4a1e-9b08-533c3e91aa2d service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:54:21.121084 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 667/1340] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:54:20 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 22:54:25.373174 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60553b4-6bfd-4624-b9d1-313daedb9c68 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:25.373727 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60553b4-6bfd-4624-b9d1-313daedb9c68 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:25.373843 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60553b4-6bfd-4624-b9d1-313daedb9c68 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:25.375225 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 674/1341] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:54:25.380755 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b6d98-3030-4aa4-ada4-ae37d010bb9f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:25.381181 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b6d98-3030-4aa4-ada4-ae37d010bb9f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:25.381311 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b6d98-3030-4aa4-ada4-ae37d010bb9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:25.458000 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e02b6d98-3030-4aa4-ada4-ae37d010bb9f None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:54:25.459776 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e02b6d98-3030-4aa4-ada4-ae37d010bb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:25.570351 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 668/1342] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:54:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:54:25.619908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60553b4-6bfd-4624-b9d1-313daedb9c68 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:54:25.626443 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d60553b4-6bfd-4624-b9d1-313daedb9c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:25.727121 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60553b4-6bfd-4624-b9d1-313daedb9c68 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:54:25.731875 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b50c1-332b-4212-aad6-8510ede5e072 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:54:25.732518 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b50c1-332b-4212-aad6-8510ede5e072 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:25.733087 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b50c1-332b-4212-aad6-8510ede5e072 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:25.733363 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b50c1-332b-4212-aad6-8510ede5e072 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:25.745334 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b50c1-332b-4212-aad6-8510ede5e072 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:54:25.746818 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b50c1-332b-4212-aad6-8510ede5e072 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:54:25.757310 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 675/1343] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:54:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:54:28.579709 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ddcb5-b4e5-41e3-b433-c20090400aef None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:28.580240 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ddcb5-b4e5-41e3-b433-c20090400aef None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:28.580240 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ddcb5-b4e5-41e3-b433-c20090400aef None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:28.581566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 669/1344] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:54:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:54:28.586676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3651a-76c4-43e0-9e2d-8a24e5e8dc5e None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:28.587099 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3651a-76c4-43e0-9e2d-8a24e5e8dc5e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:28.587260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3651a-76c4-43e0-9e2d-8a24e5e8dc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:28.615161 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-83c3651a-76c4-43e0-9e2d-8a24e5e8dc5e None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:54:28.616350 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-83c3651a-76c4-43e0-9e2d-8a24e5e8dc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:28.691474 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 676/1345] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:54:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:54:28.727962 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ddcb5-b4e5-41e3-b433-c20090400aef None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:54:28.739361 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ddcb5-b4e5-41e3-b433-c20090400aef None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:54:28.743068 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881c9c4-b0af-4baa-8bbb-543349bd8532 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:54:28.743702 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881c9c4-b0af-4baa-8bbb-543349bd8532 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:28.743983 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881c9c4-b0af-4baa-8bbb-543349bd8532 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:28.744256 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881c9c4-b0af-4baa-8bbb-543349bd8532 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:28.754789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881c9c4-b0af-4baa-8bbb-543349bd8532 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:54:28.756233 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881c9c4-b0af-4baa-8bbb-543349bd8532 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:54:28.766044 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 670/1346] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:54:28 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:54:33.092355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e63b6-053e-4baf-8267-4b73711419f3 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:33.092910 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e63b6-053e-4baf-8267-4b73711419f3 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:33.092910 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e63b6-053e-4baf-8267-4b73711419f3 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:33.094250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 677/1347] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:54:33.099494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046112ec-ac95-4ff4-afd6-24d6479089d3 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:33.099939 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046112ec-ac95-4ff4-afd6-24d6479089d3 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:33.100034 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046112ec-ac95-4ff4-afd6-24d6479089d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:33.117879 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-046112ec-ac95-4ff4-afd6-24d6479089d3 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:54:33.119101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-046112ec-ac95-4ff4-afd6-24d6479089d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:33.127632 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 671/1348] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:54:33.164732 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e63b6-053e-4baf-8267-4b73711419f3 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:54:33.175252 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e63b6-053e-4baf-8267-4b73711419f3 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:54:33.178703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab63f6-17d3-4759-badf-1add42029bef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:54:33.179331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab63f6-17d3-4759-badf-1add42029bef service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:33.179646 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab63f6-17d3-4759-badf-1add42029bef service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:33.179919 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab63f6-17d3-4759-badf-1add42029bef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:33.189422 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab63f6-17d3-4759-badf-1add42029bef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:54:33.190897 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab63f6-17d3-4759-badf-1add42029bef service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:54:33.200662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 678/1349] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:54:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:54:38.539988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f04c2-76b1-4ee0-a337-85c51df0f26e None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:38.540505 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f04c2-76b1-4ee0-a337-85c51df0f26e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:38.540505 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f04c2-76b1-4ee0-a337-85c51df0f26e None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:38.541927 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 672/1350] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 9 22:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:54:38.548992 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799379ee-67d7-451c-a186-90805dd9295b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:38.550263 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799379ee-67d7-451c-a186-90805dd9295b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:38.550772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799379ee-67d7-451c-a186-90805dd9295b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:38.649041 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-799379ee-67d7-451c-a186-90805dd9295b None None] MFA Rules not processed for user `cf9bc2fa959d4706b9f9e87509829059`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:54:38.653523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-799379ee-67d7-451c-a186-90805dd9295b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:38.796566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 679/1351] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 9 22:54:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:54:42.829938 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70cb57f-7097-4096-9a54-431c59db26e8 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:42.830583 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70cb57f-7097-4096-9a54-431c59db26e8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:42.830583 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70cb57f-7097-4096-9a54-431c59db26e8 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:42.832138 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 673/1352] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 9 22:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:54:42.837192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ea851-dc6f-4f97-9b3d-65ac41c6a530 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:42.838185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ea851-dc6f-4f97-9b3d-65ac41c6a530 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:42.838496 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ea851-dc6f-4f97-9b3d-65ac41c6a530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:42.857099 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-da5ea851-dc6f-4f97-9b3d-65ac41c6a530 None None] MFA Rules not processed for user `cf9bc2fa959d4706b9f9e87509829059`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:54:42.858714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-da5ea851-dc6f-4f97-9b3d-65ac41c6a530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:42.867774 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 680/1353] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 9 22:54:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:54:44.590282 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5dddd-c09c-4299-b31f-29cdf1643a80 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:44.590806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5dddd-c09c-4299-b31f-29cdf1643a80 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:44.590806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5dddd-c09c-4299-b31f-29cdf1643a80 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:44.592156 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 674/1354] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 9 22:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:54:44.596919 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa88497-156f-4cc9-b0d7-b1c3c33066ab None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:44.597339 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa88497-156f-4cc9-b0d7-b1c3c33066ab None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:44.597480 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa88497-156f-4cc9-b0d7-b1c3c33066ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:44.616054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-2fa88497-156f-4cc9-b0d7-b1c3c33066ab None None] MFA Rules not processed for user `cf9bc2fa959d4706b9f9e87509829059`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:54:44.617266 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-2fa88497-156f-4cc9-b0d7-b1c3c33066ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:44.626714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 681/1355] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 9 22:54:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:54:48.473258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf2a27-b0e3-4556-b2d4-031a21656a78 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:48.473782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf2a27-b0e3-4556-b2d4-031a21656a78 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:48.473782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf2a27-b0e3-4556-b2d4-031a21656a78 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:48.478616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 675/1356] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:54:48.480946 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f98dea-3cbf-4f8e-930e-1983e4d1dc1f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:48.481370 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f98dea-3cbf-4f8e-930e-1983e4d1dc1f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:48.481501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f98dea-3cbf-4f8e-930e-1983e4d1dc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:48.499879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-57f98dea-3cbf-4f8e-930e-1983e4d1dc1f None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:54:48.501060 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-57f98dea-3cbf-4f8e-930e-1983e4d1dc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:48.509510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 682/1357] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:54:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:54:48.556224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf4187-a4b4-4683-99cb-3a192fc00293 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:48.557139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf4187-a4b4-4683-99cb-3a192fc00293 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:48.557498 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf4187-a4b4-4683-99cb-3a192fc00293 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:48.558990 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 676/1358] 10.222.0.23 () {54 vars in 961 bytes} [Mon May 9 22:54:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:54:48.564289 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e7f6d-67e8-4790-b818-72cd2b971930 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:48.564710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e7f6d-67e8-4790-b818-72cd2b971930 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:48.565291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e7f6d-67e8-4790-b818-72cd2b971930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:48.614062 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-9e5e7f6d-67e8-4790-b818-72cd2b971930 None None] MFA Rules not processed for user `02fcd7cc86134ce488c7dd75dfb174b8`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:54:48.615631 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9e5e7f6d-67e8-4790-b818-72cd2b971930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:48.701343 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 683/1359] 10.222.0.23 () {58 vars in 1058 bytes} [Mon May 9 22:54:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:54:48.709164 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf4187-a4b4-4683-99cb-3a192fc00293 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:54:48.724389 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf4187-a4b4-4683-99cb-3a192fc00293 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:54:48.729878 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940cfb5-3163-4cd8-8934-332236a85057 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:54:48.730547 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940cfb5-3163-4cd8-8934-332236a85057 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:48.730820 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940cfb5-3163-4cd8-8934-332236a85057 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:48.731091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940cfb5-3163-4cd8-8934-332236a85057 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:48.741200 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940cfb5-3163-4cd8-8934-332236a85057 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:54:48.743060 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940cfb5-3163-4cd8-8934-332236a85057 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:54:48.752647 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 677/1360] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:54:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:54:51.193732 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762c329-a75a-4134-b538-7bedf2eb0a87 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:51.194329 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762c329-a75a-4134-b538-7bedf2eb0a87 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:51.194329 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762c329-a75a-4134-b538-7bedf2eb0a87 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:51.195736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 684/1361] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 22:54:51.201349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abdfd8-d43e-4da9-b35c-81acc62fbab0 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:51.201788 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abdfd8-d43e-4da9-b35c-81acc62fbab0 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:51.201975 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abdfd8-d43e-4da9-b35c-81acc62fbab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:51.219386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e0abdfd8-d43e-4da9-b35c-81acc62fbab0 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:54:51.220551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e0abdfd8-d43e-4da9-b35c-81acc62fbab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:51.228354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 678/1362] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:54:51.274217 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762c329-a75a-4134-b538-7bedf2eb0a87 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:54:51.289695 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762c329-a75a-4134-b538-7bedf2eb0a87 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:54:51.295839 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fd2be-89da-4063-b6cc-a47a30824585 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:54:51.296480 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fd2be-89da-4063-b6cc-a47a30824585 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:51.296759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fd2be-89da-4063-b6cc-a47a30824585 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:51.297033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fd2be-89da-4063-b6cc-a47a30824585 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:51.305586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fd2be-89da-4063-b6cc-a47a30824585 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:54:51.307707 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fd2be-89da-4063-b6cc-a47a30824585 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:54:51.317343 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 685/1363] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:54:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:54:53.890401 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c52b14-5b4e-4c6f-bb47-fe67d846a8c5 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:53.890885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c52b14-5b4e-4c6f-bb47-fe67d846a8c5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:53.890885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c52b14-5b4e-4c6f-bb47-fe67d846a8c5 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:53.896426 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 679/1364] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:54:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:54:53.901696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c0461-23e4-4f51-8719-b044ca1724b0 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:53.902214 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c0461-23e4-4f51-8719-b044ca1724b0 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:53.902299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c0461-23e4-4f51-8719-b044ca1724b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:53.922207 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-275c0461-23e4-4f51-8719-b044ca1724b0 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:54:53.923433 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-275c0461-23e4-4f51-8719-b044ca1724b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:53.932286 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 686/1365] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:54:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:54:53.976360 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22692ed-3464-410c-abc6-2e103c441b13 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:53.976639 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22692ed-3464-410c-abc6-2e103c441b13 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:53.976908 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22692ed-3464-410c-abc6-2e103c441b13 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:53.978795 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 680/1366] 10.222.0.23 () {54 vars in 961 bytes} [Mon May 9 22:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:54:53.984500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2f59c-479e-4a79-a354-618c24b6535f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:53.984783 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2f59c-479e-4a79-a354-618c24b6535f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:53.985054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2f59c-479e-4a79-a354-618c24b6535f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:54.003250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-fba2f59c-479e-4a79-a354-618c24b6535f None None] MFA Rules not processed for user `02fcd7cc86134ce488c7dd75dfb174b8`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:54:54.004411 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-fba2f59c-479e-4a79-a354-618c24b6535f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:54.013674 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 687/1367] 10.222.0.23 () {58 vars in 1058 bytes} [Mon May 9 22:54:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:54:54.021319 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22692ed-3464-410c-abc6-2e103c441b13 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:54:54.035406 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22692ed-3464-410c-abc6-2e103c441b13 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:54:54.041272 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a1668a-356c-4e6d-905f-c5d9f6deda41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:54:54.041884 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1668a-356c-4e6d-905f-c5d9f6deda41 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:54.042186 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1668a-356c-4e6d-905f-c5d9f6deda41 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:54.042457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1668a-356c-4e6d-905f-c5d9f6deda41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:54.050405 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a1668a-356c-4e6d-905f-c5d9f6deda41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:54:54.051978 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a1668a-356c-4e6d-905f-c5d9f6deda41 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:54:54.061294 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 681/1368] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:54:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:54:56.351748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376936d-2e3b-4e92-82ce-7c6f86644b94 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:56.352348 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376936d-2e3b-4e92-82ce-7c6f86644b94 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:56.352348 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376936d-2e3b-4e92-82ce-7c6f86644b94 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:56.353690 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 688/1369] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:54:56.359387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acadfcb7-c2ac-4913-b069-ae4ab07851f0 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:56.360178 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acadfcb7-c2ac-4913-b069-ae4ab07851f0 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:56.360457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acadfcb7-c2ac-4913-b069-ae4ab07851f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:56.379063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-acadfcb7-c2ac-4913-b069-ae4ab07851f0 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:54:56.380333 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-acadfcb7-c2ac-4913-b069-ae4ab07851f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:56.388915 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 682/1370] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:54:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:54:56.431819 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376936d-2e3b-4e92-82ce-7c6f86644b94 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:54:56.446217 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376936d-2e3b-4e92-82ce-7c6f86644b94 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:54:56.452440 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6202395-c656-4355-8fad-75e346174e33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:54:56.453196 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6202395-c656-4355-8fad-75e346174e33 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:56.453559 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6202395-c656-4355-8fad-75e346174e33 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:56.453899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6202395-c656-4355-8fad-75e346174e33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:56.463445 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6202395-c656-4355-8fad-75e346174e33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:54:56.465083 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6202395-c656-4355-8fad-75e346174e33 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:54:56.476117 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 689/1371] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:54:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:54:56.659411 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe727b-c847-4938-a683-6072cffdd508 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:56.659769 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe727b-c847-4938-a683-6072cffdd508 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:56.659965 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe727b-c847-4938-a683-6072cffdd508 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:56.661582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 683/1372] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:54:56.668265 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82d143-f1a3-484a-a5a0-0ccfd1710c27 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:56.668551 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82d143-f1a3-484a-a5a0-0ccfd1710c27 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:56.669029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82d143-f1a3-484a-a5a0-0ccfd1710c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:56.690184 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-be82d143-f1a3-484a-a5a0-0ccfd1710c27 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:54:56.691464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-be82d143-f1a3-484a-a5a0-0ccfd1710c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:56.700980 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 690/1373] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:54:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:54:57.033621 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4bc41-4094-4a9c-926d-b68d67697b88 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:57.033895 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4bc41-4094-4a9c-926d-b68d67697b88 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:57.034197 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4bc41-4094-4a9c-926d-b68d67697b88 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:57.036146 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 684/1374] 10.222.0.23 () {54 vars in 963 bytes} [Mon May 9 22:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:54:57.040963 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72c187-2bc0-42dc-abff-742283c3236e None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:57.041205 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72c187-2bc0-42dc-abff-742283c3236e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:57.041485 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72c187-2bc0-42dc-abff-742283c3236e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:57.093927 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-be72c187-2bc0-42dc-abff-742283c3236e None None] MFA Rules not processed for user `5fe668a2cee040c7a56d1f8685744cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:54:57.095256 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-be72c187-2bc0-42dc-abff-742283c3236e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:57.156080 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 691/1375] 10.222.0.23 () {58 vars in 1060 bytes} [Mon May 9 22:54:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:54:57.163832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4bc41-4094-4a9c-926d-b68d67697b88 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:54:57.174554 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4bc41-4094-4a9c-926d-b68d67697b88 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:54:57.178191 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86385cd-b889-43ff-8d8e-18f2d9a7dd6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:54:57.178828 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86385cd-b889-43ff-8d8e-18f2d9a7dd6f service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:57.179109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86385cd-b889-43ff-8d8e-18f2d9a7dd6f service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:57.179380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86385cd-b889-43ff-8d8e-18f2d9a7dd6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:57.189232 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86385cd-b889-43ff-8d8e-18f2d9a7dd6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:54:57.190714 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86385cd-b889-43ff-8d8e-18f2d9a7dd6f service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:54:57.200980 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 685/1376] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:54:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 22:54:58.749423 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8a12f-9ec5-4102-8a05-1e3aafe6cfaf None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:58.750022 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8a12f-9ec5-4102-8a05-1e3aafe6cfaf None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:58.750022 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8a12f-9ec5-4102-8a05-1e3aafe6cfaf None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:58.751436 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 692/1377] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:54:58.757856 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392710b-e678-4947-aa9c-3a934fb6f5f4 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:58.758417 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392710b-e678-4947-aa9c-3a934fb6f5f4 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:58.758499 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392710b-e678-4947-aa9c-3a934fb6f5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:58.778241 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-0392710b-e678-4947-aa9c-3a934fb6f5f4 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:54:58.779450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0392710b-e678-4947-aa9c-3a934fb6f5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:54:58.787504 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 686/1378] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:54:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:54:58.832840 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8a12f-9ec5-4102-8a05-1e3aafe6cfaf None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:54:58.847220 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8a12f-9ec5-4102-8a05-1e3aafe6cfaf None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:54:58.852774 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50de558-96e5-4576-8987-f5597bf9b223 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:54:58.853446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50de558-96e5-4576-8987-f5597bf9b223 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:54:58.853726 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50de558-96e5-4576-8987-f5597bf9b223 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:54:58.854001 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50de558-96e5-4576-8987-f5597bf9b223 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:54:58.863243 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50de558-96e5-4576-8987-f5597bf9b223 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:54:58.864864 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50de558-96e5-4576-8987-f5597bf9b223 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:54:58.881109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 693/1379] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:54:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 22:55:01.215556 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f713a2-b9d8-43e6-8041-13e637892533 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:01.216085 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f713a2-b9d8-43e6-8041-13e637892533 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:01.216085 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f713a2-b9d8-43e6-8041-13e637892533 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:01.217436 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 687/1380] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:01.222575 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82798d12-ddc7-46e9-8e6a-0f379357f7ad None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:01.223009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82798d12-ddc7-46e9-8e6a-0f379357f7ad None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:01.223116 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82798d12-ddc7-46e9-8e6a-0f379357f7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:01.240770 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-82798d12-ddc7-46e9-8e6a-0f379357f7ad None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:01.241862 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-82798d12-ddc7-46e9-8e6a-0f379357f7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:01.249579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 694/1381] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:55:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:55:01.292803 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f713a2-b9d8-43e6-8041-13e637892533 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:55:01.307412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f713a2-b9d8-43e6-8041-13e637892533 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:55:01.313180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327e4b2-51f5-4161-b505-d03197e3127c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:55:01.314033 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327e4b2-51f5-4161-b505-d03197e3127c service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:01.314553 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327e4b2-51f5-4161-b505-d03197e3127c service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:01.315101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327e4b2-51f5-4161-b505-d03197e3127c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:01.324077 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327e4b2-51f5-4161-b505-d03197e3127c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:55:01.325851 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327e4b2-51f5-4161-b505-d03197e3127c service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:55:01.335446 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 688/1382] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:55:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:55:03.635044 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd724dd8-f3e6-4445-a5b2-1a7661b82922 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:03.635919 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd724dd8-f3e6-4445-a5b2-1a7661b82922 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:03.636350 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd724dd8-f3e6-4445-a5b2-1a7661b82922 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:03.640239 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 695/1383] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:55:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:03.643288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d773275-ad98-4758-8979-3a3d59589083 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:03.643747 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d773275-ad98-4758-8979-3a3d59589083 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:03.643854 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d773275-ad98-4758-8979-3a3d59589083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:03.661656 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-1d773275-ad98-4758-8979-3a3d59589083 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:03.662892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1d773275-ad98-4758-8979-3a3d59589083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:03.674785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 689/1384] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:55:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:55:03.716303 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd724dd8-f3e6-4445-a5b2-1a7661b82922 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:55:03.732182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd724dd8-f3e6-4445-a5b2-1a7661b82922 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:55:03.738213 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653429c-213b-478e-b8ac-b4805e48d58a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:55:03.738872 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653429c-213b-478e-b8ac-b4805e48d58a service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:03.739150 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653429c-213b-478e-b8ac-b4805e48d58a service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:03.739423 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653429c-213b-478e-b8ac-b4805e48d58a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:03.748764 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6653429c-213b-478e-b8ac-b4805e48d58a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:55:03.750370 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6653429c-213b-478e-b8ac-b4805e48d58a service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:55:03.760252 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 696/1385] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:55:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 22:55:05.993875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6655f79-8092-44e8-8529-109c6533ebf9 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:05.994411 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6655f79-8092-44e8-8529-109c6533ebf9 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:05.994592 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6655f79-8092-44e8-8529-109c6533ebf9 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:05.996384 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 690/1386] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:55:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:06.003698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2d264-3436-4d8c-8699-8118fc05dfda None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:06.004324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2d264-3436-4d8c-8699-8118fc05dfda None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:06.004513 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2d264-3436-4d8c-8699-8118fc05dfda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:06.027284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f7b2d264-3436-4d8c-8699-8118fc05dfda None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:06.028772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f7b2d264-3436-4d8c-8699-8118fc05dfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:06.038448 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 697/1387] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:55:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:55:06.082465 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6655f79-8092-44e8-8529-109c6533ebf9 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:55:06.096861 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6655f79-8092-44e8-8529-109c6533ebf9 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:55:06.102386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca3885-f2d1-4d97-a295-c2c777f9f7b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:55:06.103025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca3885-f2d1-4d97-a295-c2c777f9f7b9 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:06.103310 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca3885-f2d1-4d97-a295-c2c777f9f7b9 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:06.103581 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca3885-f2d1-4d97-a295-c2c777f9f7b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:06.115059 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca3885-f2d1-4d97-a295-c2c777f9f7b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:55:06.116640 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca3885-f2d1-4d97-a295-c2c777f9f7b9 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:55:06.126909 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 691/1388] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:55:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:55:08.540660 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f66f5-6882-4ae6-a78c-030d0c3abc8b None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:08.541210 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f66f5-6882-4ae6-a78c-030d0c3abc8b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:08.541413 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f66f5-6882-4ae6-a78c-030d0c3abc8b None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:08.543339 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 698/1389] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:55:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:08.548692 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ba3b0-4ab4-4577-a373-65b981266eaf None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:08.549122 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ba3b0-4ab4-4577-a373-65b981266eaf None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:08.549282 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ba3b0-4ab4-4577-a373-65b981266eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:08.569078 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b21ba3b0-4ab4-4577-a373-65b981266eaf None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:08.570386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b21ba3b0-4ab4-4577-a373-65b981266eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:08.579164 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 692/1390] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:55:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:55:08.630749 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f66f5-6882-4ae6-a78c-030d0c3abc8b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:55:08.649381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f66f5-6882-4ae6-a78c-030d0c3abc8b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:55:08.655892 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41341b-607d-4915-abed-d36b6ebe9451 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:55:08.656711 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41341b-607d-4915-abed-d36b6ebe9451 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:08.657023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41341b-607d-4915-abed-d36b6ebe9451 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:08.657355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41341b-607d-4915-abed-d36b6ebe9451 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:08.667859 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41341b-607d-4915-abed-d36b6ebe9451 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:55:08.669842 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41341b-607d-4915-abed-d36b6ebe9451 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:55:08.681901 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 699/1391] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:55:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:55:10.995516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e459227-79bc-47a0-b3ad-59073d499552 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:10.996130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e459227-79bc-47a0-b3ad-59073d499552 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:10.996130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e459227-79bc-47a0-b3ad-59073d499552 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:10.997326 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 693/1392] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:11.002586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048c9d6-c6f7-4c3c-8ae6-72f292ee6c2b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:11.003027 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048c9d6-c6f7-4c3c-8ae6-72f292ee6c2b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:11.003129 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048c9d6-c6f7-4c3c-8ae6-72f292ee6c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:11.021147 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-7048c9d6-c6f7-4c3c-8ae6-72f292ee6c2b None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:11.022360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7048c9d6-c6f7-4c3c-8ae6-72f292ee6c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:11.030679 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 700/1393] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:55:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:55:11.073083 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e459227-79bc-47a0-b3ad-59073d499552 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:55:11.087670 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e459227-79bc-47a0-b3ad-59073d499552 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:55:11.093083 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae168b3e-3e61-4cb2-b2e4-c8d397f9ec45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:55:11.093712 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae168b3e-3e61-4cb2-b2e4-c8d397f9ec45 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:11.093991 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae168b3e-3e61-4cb2-b2e4-c8d397f9ec45 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:11.094295 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae168b3e-3e61-4cb2-b2e4-c8d397f9ec45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:11.102956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae168b3e-3e61-4cb2-b2e4-c8d397f9ec45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:55:11.104525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae168b3e-3e61-4cb2-b2e4-c8d397f9ec45 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:55:11.114066 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 694/1394] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:55:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:55:13.062857 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c573d5d-1507-4a4d-b74d-03f31836d6ee None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:13.063426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c573d5d-1507-4a4d-b74d-03f31836d6ee None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:13.063426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c573d5d-1507-4a4d-b74d-03f31836d6ee None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:13.064753 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 701/1395] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:13.070028 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7fedf-831c-4ca1-b0e6-46955f8bba66 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:13.070457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7fedf-831c-4ca1-b0e6-46955f8bba66 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:13.070804 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7fedf-831c-4ca1-b0e6-46955f8bba66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:13.088437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-0ab7fedf-831c-4ca1-b0e6-46955f8bba66 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:13.089614 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0ab7fedf-831c-4ca1-b0e6-46955f8bba66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:13.097405 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 695/1396] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:55:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:55:13.155761 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21560c-2447-4c51-be61-3fe050d5e1b4 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:13.156161 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21560c-2447-4c51-be61-3fe050d5e1b4 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:13.156446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21560c-2447-4c51-be61-3fe050d5e1b4 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:13.157827 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 702/1397] 10.222.0.23 () {54 vars in 962 bytes} [Mon May 9 22:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:13.162875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d5253-2c08-4715-9860-dbd2ce2736c3 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:13.163166 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d5253-2c08-4715-9860-dbd2ce2736c3 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:13.163442 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d5253-2c08-4715-9860-dbd2ce2736c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:13.214933 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-4f8d5253-2c08-4715-9860-dbd2ce2736c3 None None] MFA Rules not processed for user `a81caf0558f2449cbbc8f4cd9446f363`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:13.216175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-4f8d5253-2c08-4715-9860-dbd2ce2736c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:13.276015 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 696/1398] 10.222.0.23 () {58 vars in 1059 bytes} [Mon May 9 22:55:13 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:55:13.283574 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21560c-2447-4c51-be61-3fe050d5e1b4 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:55:13.294199 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21560c-2447-4c51-be61-3fe050d5e1b4 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:55:13.297699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d564ba-ab8c-44f5-ae2a-2483e5588737 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81caf0558f2449cbbc8f4cd9446f363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:55:13.298362 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d564ba-ab8c-44f5-ae2a-2483e5588737 service glance] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:13.298643 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d564ba-ab8c-44f5-ae2a-2483e5588737 service glance] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:13.298917 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d564ba-ab8c-44f5-ae2a-2483e5588737 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:13.307767 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d564ba-ab8c-44f5-ae2a-2483e5588737 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:55:13.309176 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d564ba-ab8c-44f5-ae2a-2483e5588737 service glance] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:55:13.318659 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 703/1399] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:55:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:55:13.392391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91ee3b-a7f0-47b5-aac3-f6961e9736cc None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:13.392711 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91ee3b-a7f0-47b5-aac3-f6961e9736cc None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:13.392992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91ee3b-a7f0-47b5-aac3-f6961e9736cc None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:13.394417 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 697/1400] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:13.400052 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c4fc3-24b2-4536-88cd-5267bec3541d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:13.400326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c4fc3-24b2-4536-88cd-5267bec3541d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:13.400602 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c4fc3-24b2-4536-88cd-5267bec3541d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:13.418346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-561c4fc3-24b2-4536-88cd-5267bec3541d None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:13.419530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-561c4fc3-24b2-4536-88cd-5267bec3541d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:13.427370 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 704/1401] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:55:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:55:13.470013 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91ee3b-a7f0-47b5-aac3-f6961e9736cc None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:55:13.485985 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91ee3b-a7f0-47b5-aac3-f6961e9736cc None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:55:13.491501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51274ca-fe38-4969-9a6a-b14b5dfbdf9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:55:13.492166 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51274ca-fe38-4969-9a6a-b14b5dfbdf9a service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:13.492457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51274ca-fe38-4969-9a6a-b14b5dfbdf9a service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:13.492729 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51274ca-fe38-4969-9a6a-b14b5dfbdf9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:13.501557 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51274ca-fe38-4969-9a6a-b14b5dfbdf9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:55:13.503174 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51274ca-fe38-4969-9a6a-b14b5dfbdf9a service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:55:13.512513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 698/1402] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:55:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:55:13.800862 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6cd7b-60fe-45c5-9e9a-1abbec17bb93 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:13.801136 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6cd7b-60fe-45c5-9e9a-1abbec17bb93 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:13.801412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6cd7b-60fe-45c5-9e9a-1abbec17bb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:13.847413 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f0c6cd7b-60fe-45c5-9e9a-1abbec17bb93 None None] MFA Rules not processed for user `1a6f50dcebf146baae8a9f0b5b112795`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:13.848502 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f0c6cd7b-60fe-45c5-9e9a-1abbec17bb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:13.910183 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 705/1403] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 9 22:55:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:55:13.952369 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95d950-7db9-41c7-a8b3-c7d51975b620 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:13.952654 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95d950-7db9-41c7-a8b3-c7d51975b620 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:13.952927 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95d950-7db9-41c7-a8b3-c7d51975b620 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:13.954326 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 699/1404] 10.222.0.23 () {54 vars in 942 bytes} [Mon May 9 22:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:13.960843 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcef9d3-72d9-46bf-92d7-44763feb61f1 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:13.961203 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcef9d3-72d9-46bf-92d7-44763feb61f1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:13.961547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcef9d3-72d9-46bf-92d7-44763feb61f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:14.009638 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-cbcef9d3-72d9-46bf-92d7-44763feb61f1 None None] MFA Rules not processed for user `8cb6c294ce1842b68a6bc73ac381ed6b`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:14.010860 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-cbcef9d3-72d9-46bf-92d7-44763feb61f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:14.079370 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 706/1405] 10.222.0.23 () {58 vars in 1039 bytes} [Mon May 9 22:55:13 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:55:14.088641 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95d950-7db9-41c7-a8b3-c7d51975b620 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:55:14.101610 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95d950-7db9-41c7-a8b3-c7d51975b620 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:55:14.106928 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a168b4-d2ad-4f75-ac1c-eef84ee01496 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cb6c294ce1842b68a6bc73ac381ed6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:55:14.107554 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a168b4-d2ad-4f75-ac1c-eef84ee01496 service swift] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:14.108088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a168b4-d2ad-4f75-ac1c-eef84ee01496 service swift] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:14.108357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a168b4-d2ad-4f75-ac1c-eef84ee01496 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:14.116334 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a168b4-d2ad-4f75-ac1c-eef84ee01496 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:55:14.117925 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a168b4-d2ad-4f75-ac1c-eef84ee01496 service swift] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:55:14.126763 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 700/1406] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 9 22:55:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 22:55:15.783510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f43d40-6e62-4cfb-bc6c-92f057bd06d6 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:15.784062 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f43d40-6e62-4cfb-bc6c-92f057bd06d6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:15.784062 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f43d40-6e62-4cfb-bc6c-92f057bd06d6 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:15.785679 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 707/1407] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:55:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:15.791038 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c4199-7592-4291-ba03-481b4c992a05 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:15.791814 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c4199-7592-4291-ba03-481b4c992a05 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:15.792091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c4199-7592-4291-ba03-481b4c992a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:15.810394 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-1b5c4199-7592-4291-ba03-481b4c992a05 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:15.811631 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1b5c4199-7592-4291-ba03-481b4c992a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:15.821987 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 701/1408] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:55:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:55:15.864470 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f43d40-6e62-4cfb-bc6c-92f057bd06d6 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:55:15.881333 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f43d40-6e62-4cfb-bc6c-92f057bd06d6 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:55:15.888662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a992124-63fd-4767-9122-763d0ab90c42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:55:15.888662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a992124-63fd-4767-9122-763d0ab90c42 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:15.888662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a992124-63fd-4767-9122-763d0ab90c42 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:15.889006 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a992124-63fd-4767-9122-763d0ab90c42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:15.897681 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a992124-63fd-4767-9122-763d0ab90c42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:55:15.899846 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a992124-63fd-4767-9122-763d0ab90c42 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:55:15.909348 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 708/1409] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:55:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:55:37.348025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987e77e-fd79-416a-9ee5-8512cdae0e57 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:37.348664 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987e77e-fd79-416a-9ee5-8512cdae0e57 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:37.348664 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987e77e-fd79-416a-9ee5-8512cdae0e57 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:37.349902 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 702/1410] 10.222.0.23 () {54 vars in 918 bytes} [Mon May 9 22:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:37.356096 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446bd24-943d-432e-a46f-1fd4a4872f96 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:37.356580 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446bd24-943d-432e-a46f-1fd4a4872f96 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:37.356689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446bd24-943d-432e-a46f-1fd4a4872f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:37.375342 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5446bd24-943d-432e-a46f-1fd4a4872f96 None None] MFA Rules not processed for user `cf9bc2fa959d4706b9f9e87509829059`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:37.376541 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5446bd24-943d-432e-a46f-1fd4a4872f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:37.385601 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 709/1411] 10.222.0.23 () {58 vars in 1015 bytes} [Mon May 9 22:55:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:55:37.399974 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09db01-8a15-435b-9a63-c5e212056f85 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:37.400254 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09db01-8a15-435b-9a63-c5e212056f85 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:37.400525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09db01-8a15-435b-9a63-c5e212056f85 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:37.401840 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 703/1412] 10.222.0.23 () {54 vars in 918 bytes} [Mon May 9 22:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:37.406436 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bc5ed-bbc3-46ea-9d27-8d9d9c584876 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:37.406713 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bc5ed-bbc3-46ea-9d27-8d9d9c584876 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:37.406985 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bc5ed-bbc3-46ea-9d27-8d9d9c584876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:37.424716 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-035bc5ed-bbc3-46ea-9d27-8d9d9c584876 None None] MFA Rules not processed for user `02fcd7cc86134ce488c7dd75dfb174b8`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:37.425899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-035bc5ed-bbc3-46ea-9d27-8d9d9c584876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:37.435358 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 710/1413] 10.222.0.23 () {58 vars in 1015 bytes} [Mon May 9 22:55:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:55:40.082630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbb269-627b-4fb0-ae65-256d8ab2c049 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:40.083204 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbb269-627b-4fb0-ae65-256d8ab2c049 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:40.083204 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbb269-627b-4fb0-ae65-256d8ab2c049 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:40.084558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 22:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:40.089887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4b924-386d-4207-920a-fc0027c76bda None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:40.090327 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4b924-386d-4207-920a-fc0027c76bda None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:40.090515 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4b924-386d-4207-920a-fc0027c76bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:40.108195 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-47e4b924-386d-4207-920a-fc0027c76bda None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:40.109588 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-47e4b924-386d-4207-920a-fc0027c76bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:40.118083 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 22:55:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:55:40.159026 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbb269-627b-4fb0-ae65-256d8ab2c049 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:55:40.173337 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbb269-627b-4fb0-ae65-256d8ab2c049 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:55:40.178648 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53456a43-6416-465d-979f-63aa65040b43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:55:40.179272 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53456a43-6416-465d-979f-63aa65040b43 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:40.179550 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53456a43-6416-465d-979f-63aa65040b43 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:40.179823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53456a43-6416-465d-979f-63aa65040b43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:40.188221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53456a43-6416-465d-979f-63aa65040b43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:55:40.189785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53456a43-6416-465d-979f-63aa65040b43 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:55:40.198912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 705/1416] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:55:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:55:41.187805 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fad6a0-be94-498f-b170-3de9c0058fad None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:41.188362 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fad6a0-be94-498f-b170-3de9c0058fad None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:41.188362 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fad6a0-be94-498f-b170-3de9c0058fad None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:41.189706 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 22:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:41.194691 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555acb6-5a0f-42af-ac66-0db511bfd4b6 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:41.195132 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555acb6-5a0f-42af-ac66-0db511bfd4b6 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:41.195239 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555acb6-5a0f-42af-ac66-0db511bfd4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:41.213534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-6555acb6-5a0f-42af-ac66-0db511bfd4b6 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:41.214749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-6555acb6-5a0f-42af-ac66-0db511bfd4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:41.222359 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 22:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:55:41.379713 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fad6a0-be94-498f-b170-3de9c0058fad None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:55:41.394311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fad6a0-be94-498f-b170-3de9c0058fad None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:55:41.399590 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d69f9d-50c7-4633-800e-8800e22efa92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:55:41.400215 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d69f9d-50c7-4633-800e-8800e22efa92 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:41.400497 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d69f9d-50c7-4633-800e-8800e22efa92 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:41.400771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d69f9d-50c7-4633-800e-8800e22efa92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:41.408573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d69f9d-50c7-4633-800e-8800e22efa92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:55:41.410164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d69f9d-50c7-4633-800e-8800e22efa92 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:55:41.419250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 713/1419] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:55:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:55:42.376735 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741038a6-3c71-40d9-b7c8-cc68c1965ca9 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:42.377321 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741038a6-3c71-40d9-b7c8-cc68c1965ca9 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:42.377321 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741038a6-3c71-40d9-b7c8-cc68c1965ca9 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:42.378690 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 22:55:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:42.384191 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2422910-e1f8-4b57-9b8c-c67cb4dd11d3 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:42.384662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2422910-e1f8-4b57-9b8c-c67cb4dd11d3 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:42.384776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2422910-e1f8-4b57-9b8c-c67cb4dd11d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:42.402452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f2422910-e1f8-4b57-9b8c-c67cb4dd11d3 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:42.403648 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f2422910-e1f8-4b57-9b8c-c67cb4dd11d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:42.411340 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 22:55:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:55:42.567427 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741038a6-3c71-40d9-b7c8-cc68c1965ca9 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:55:42.581457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741038a6-3c71-40d9-b7c8-cc68c1965ca9 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:55:42.586870 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3ec14-4c6c-41ab-8d67-218fb0da09c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:55:42.587500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3ec14-4c6c-41ab-8d67-218fb0da09c6 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:42.587778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3ec14-4c6c-41ab-8d67-218fb0da09c6 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:42.588051 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3ec14-4c6c-41ab-8d67-218fb0da09c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:42.595874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf3ec14-4c6c-41ab-8d67-218fb0da09c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:55:42.597435 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf3ec14-4c6c-41ab-8d67-218fb0da09c6 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:55:42.607084 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 708/1422] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:55:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:55:43.546605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bfef0e-9165-4e2b-a488-743d459b8401 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:43.547158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bfef0e-9165-4e2b-a488-743d459b8401 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:43.547260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bfef0e-9165-4e2b-a488-743d459b8401 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:43.548588 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 22:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:43.553511 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ff614-d5be-4aa4-9b59-696fd8bb585f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:43.553914 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ff614-d5be-4aa4-9b59-696fd8bb585f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:43.554067 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ff614-d5be-4aa4-9b59-696fd8bb585f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:43.572774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-3b7ff614-d5be-4aa4-9b59-696fd8bb585f None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:43.573959 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3b7ff614-d5be-4aa4-9b59-696fd8bb585f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:43.581757 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 22:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:55:43.728992 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bfef0e-9165-4e2b-a488-743d459b8401 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:55:43.743848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bfef0e-9165-4e2b-a488-743d459b8401 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:55:43.749069 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6023892-d87d-4c45-aa40-587e802824d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:55:43.749694 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6023892-d87d-4c45-aa40-587e802824d1 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:43.749974 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6023892-d87d-4c45-aa40-587e802824d1 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:43.750279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6023892-d87d-4c45-aa40-587e802824d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:43.759284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6023892-d87d-4c45-aa40-587e802824d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:55:43.761183 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6023892-d87d-4c45-aa40-587e802824d1 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:55:43.771198 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 716/1425] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:55:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:55:44.708084 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3796b54-1e63-4a8c-a3e3-2c35e43e0fe6 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:44.708614 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3796b54-1e63-4a8c-a3e3-2c35e43e0fe6 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:44.708614 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3796b54-1e63-4a8c-a3e3-2c35e43e0fe6 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:44.709931 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 22:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:44.715001 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6237f-e7e8-49fd-bbc5-0f3bb9236270 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:44.715436 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6237f-e7e8-49fd-bbc5-0f3bb9236270 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:44.715606 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6237f-e7e8-49fd-bbc5-0f3bb9236270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:44.733830 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-cbd6237f-e7e8-49fd-bbc5-0f3bb9236270 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:44.735066 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-cbd6237f-e7e8-49fd-bbc5-0f3bb9236270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:44.743447 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 22:55:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:55:44.890290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3796b54-1e63-4a8c-a3e3-2c35e43e0fe6 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:55:44.904025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3796b54-1e63-4a8c-a3e3-2c35e43e0fe6 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:55:44.909252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e26ec-396d-4dcf-962d-29dd473163f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:55:44.909867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e26ec-396d-4dcf-962d-29dd473163f8 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:44.910212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e26ec-396d-4dcf-962d-29dd473163f8 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:44.910432 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e26ec-396d-4dcf-962d-29dd473163f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:44.918723 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e26ec-396d-4dcf-962d-29dd473163f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:55:44.920267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e26ec-396d-4dcf-962d-29dd473163f8 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:55:44.929228 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 711/1428] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:55:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:55:45.788708 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc576aa-3694-4af9-b309-36ce7941267e None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:45.789309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc576aa-3694-4af9-b309-36ce7941267e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:45.789309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc576aa-3694-4af9-b309-36ce7941267e None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:45.790653 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 22:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:45.795616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3105fe-eac7-4fca-bb2e-aa1f304d2d66 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:45.796024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3105fe-eac7-4fca-bb2e-aa1f304d2d66 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:45.796205 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3105fe-eac7-4fca-bb2e-aa1f304d2d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:45.814084 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-be3105fe-eac7-4fca-bb2e-aa1f304d2d66 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:45.815304 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-be3105fe-eac7-4fca-bb2e-aa1f304d2d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:45.823794 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 22:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:55:45.972273 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc576aa-3694-4af9-b309-36ce7941267e None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:55:45.986461 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc576aa-3694-4af9-b309-36ce7941267e None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:55:45.991717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e1329-f1b2-4d88-aee4-17fc62770c6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:55:45.992855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e1329-f1b2-4d88-aee4-17fc62770c6e service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:45.993130 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e1329-f1b2-4d88-aee4-17fc62770c6e service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:45.993396 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e1329-f1b2-4d88-aee4-17fc62770c6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:46.001145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e1329-f1b2-4d88-aee4-17fc62770c6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:55:46.002749 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e1329-f1b2-4d88-aee4-17fc62770c6e service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:55:46.012292 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 719/1431] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:55:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:55:46.890427 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95654ed6-b7c0-4c47-a26c-cba939d71500 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:46.890997 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95654ed6-b7c0-4c47-a26c-cba939d71500 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:46.890997 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95654ed6-b7c0-4c47-a26c-cba939d71500 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:46.892368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 22:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:46.897254 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a097d3-de08-4e01-a485-6cddd3b3c662 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:46.897669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a097d3-de08-4e01-a485-6cddd3b3c662 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:46.897790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a097d3-de08-4e01-a485-6cddd3b3c662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:46.916400 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c8a097d3-de08-4e01-a485-6cddd3b3c662 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:46.917589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c8a097d3-de08-4e01-a485-6cddd3b3c662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:46.925322 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 22:55:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 22:55:47.075709 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95654ed6-b7c0-4c47-a26c-cba939d71500 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:55:47.089643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95654ed6-b7c0-4c47-a26c-cba939d71500 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:55:47.094955 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1128b-5b86-4884-86e2-8e16a6d8b960 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:55:47.095577 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1128b-5b86-4884-86e2-8e16a6d8b960 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:47.095862 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1128b-5b86-4884-86e2-8e16a6d8b960 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:47.096130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1128b-5b86-4884-86e2-8e16a6d8b960 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:47.104548 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1128b-5b86-4884-86e2-8e16a6d8b960 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:55:47.106132 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1128b-5b86-4884-86e2-8e16a6d8b960 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:55:47.115486 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 714/1434] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:55:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:55:47.980947 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de798b-0966-4408-bda2-ebece44797a0 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:47.981488 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de798b-0966-4408-bda2-ebece44797a0 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:47.981488 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de798b-0966-4408-bda2-ebece44797a0 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:47.983041 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 22:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:47.987818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89935c36-7d37-4a32-9abe-2d60e63460c5 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:47.988266 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89935c36-7d37-4a32-9abe-2d60e63460c5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:47.988375 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89935c36-7d37-4a32-9abe-2d60e63460c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:48.005870 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-89935c36-7d37-4a32-9abe-2d60e63460c5 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:48.007089 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-89935c36-7d37-4a32-9abe-2d60e63460c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:48.015322 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 22:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:55:48.170759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de798b-0966-4408-bda2-ebece44797a0 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:55:48.185865 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de798b-0966-4408-bda2-ebece44797a0 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:55:48.191404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31f7e1-fc9c-490c-899d-42c205e7e180 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:55:48.192057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31f7e1-fc9c-490c-899d-42c205e7e180 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:48.192336 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31f7e1-fc9c-490c-899d-42c205e7e180 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:48.192608 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31f7e1-fc9c-490c-899d-42c205e7e180 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:48.201695 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31f7e1-fc9c-490c-899d-42c205e7e180 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:55:48.203308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31f7e1-fc9c-490c-899d-42c205e7e180 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:55:48.212528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 722/1437] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:55:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:55:49.086709 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc952ac8-bf16-4fde-838b-89c79faedfe3 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:49.087281 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc952ac8-bf16-4fde-838b-89c79faedfe3 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:49.087546 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc952ac8-bf16-4fde-838b-89c79faedfe3 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:49.088979 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 22:55:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:49.093940 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc9602-86d7-4303-b016-00ae38fbb76c None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:49.094426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc9602-86d7-4303-b016-00ae38fbb76c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:49.094530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc9602-86d7-4303-b016-00ae38fbb76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:49.113154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-07cc9602-86d7-4303-b016-00ae38fbb76c None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:49.114356 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-07cc9602-86d7-4303-b016-00ae38fbb76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:49.121980 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 22:55:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:55:49.273197 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc952ac8-bf16-4fde-838b-89c79faedfe3 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:55:49.287804 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc952ac8-bf16-4fde-838b-89c79faedfe3 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:55:49.293131 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa24f6a-9780-425b-a91c-9c2a9dc6a11e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:55:49.293752 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa24f6a-9780-425b-a91c-9c2a9dc6a11e service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:49.294028 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa24f6a-9780-425b-a91c-9c2a9dc6a11e service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:49.294326 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa24f6a-9780-425b-a91c-9c2a9dc6a11e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:49.302054 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa24f6a-9780-425b-a91c-9c2a9dc6a11e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:55:49.303640 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa24f6a-9780-425b-a91c-9c2a9dc6a11e service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:55:49.312639 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 717/1440] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:55:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:55:51.419702 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc823e9d-547f-47fc-936c-29bf22ce21d4 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:51.420258 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc823e9d-547f-47fc-936c-29bf22ce21d4 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:51.420258 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc823e9d-547f-47fc-936c-29bf22ce21d4 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:51.421598 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 22:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:51.426845 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd1188-c480-4557-b0a4-b5e4caaa6775 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:51.427293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd1188-c480-4557-b0a4-b5e4caaa6775 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:51.427392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd1188-c480-4557-b0a4-b5e4caaa6775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:51.445084 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-befd1188-c480-4557-b0a4-b5e4caaa6775 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:51.446289 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-befd1188-c480-4557-b0a4-b5e4caaa6775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:51.453977 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 22:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:55:51.491509 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc823e9d-547f-47fc-936c-29bf22ce21d4 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:55:51.493755 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-cc823e9d-547f-47fc-936c-29bf22ce21d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:51.506093 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc823e9d-547f-47fc-936c-29bf22ce21d4 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:55:51.509554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29be843-a0cf-42e5-866d-05cd20f498cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:55:51.510203 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29be843-a0cf-42e5-866d-05cd20f498cd service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:51.510482 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29be843-a0cf-42e5-866d-05cd20f498cd service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:51.510750 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29be843-a0cf-42e5-866d-05cd20f498cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:51.519149 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29be843-a0cf-42e5-866d-05cd20f498cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:55:51.520546 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29be843-a0cf-42e5-866d-05cd20f498cd service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:55:51.529612 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 725/1443] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:55:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:55:53.849590 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a73b8-4db3-4058-ac4b-e63c5fc92f91 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:53.850275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a73b8-4db3-4058-ac4b-e63c5fc92f91 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:53.850275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a73b8-4db3-4058-ac4b-e63c5fc92f91 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:53.852086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 22:55:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:53.857983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196c64a-1f66-4d34-a969-86286b29e3bb None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:53.858433 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196c64a-1f66-4d34-a969-86286b29e3bb None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:53.858575 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196c64a-1f66-4d34-a969-86286b29e3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:53.876397 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-6196c64a-1f66-4d34-a969-86286b29e3bb None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:53.877633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6196c64a-1f66-4d34-a969-86286b29e3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:53.885390 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 22:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:55:53.923094 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a73b8-4db3-4058-ac4b-e63c5fc92f91 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:55:53.934070 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a73b8-4db3-4058-ac4b-e63c5fc92f91 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:55:53.937885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798984a2-5bed-4735-b6e3-a64bbd2047aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:55:53.938552 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798984a2-5bed-4735-b6e3-a64bbd2047aa service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:53.938835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798984a2-5bed-4735-b6e3-a64bbd2047aa service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:53.939111 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798984a2-5bed-4735-b6e3-a64bbd2047aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:53.948468 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798984a2-5bed-4735-b6e3-a64bbd2047aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:55:53.949906 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798984a2-5bed-4735-b6e3-a64bbd2047aa service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:55:53.958892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 720/1446] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:55:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:55:56.454265 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c107a6-dfb7-4f94-a16c-943e5c82c551 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:56.454816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c107a6-dfb7-4f94-a16c-943e5c82c551 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:56.454816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c107a6-dfb7-4f94-a16c-943e5c82c551 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:56.456186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 22:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:56.461370 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae77845-a554-4187-974b-39a9ad987908 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:56.461777 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae77845-a554-4187-974b-39a9ad987908 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:56.461949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae77845-a554-4187-974b-39a9ad987908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:56.479093 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-2ae77845-a554-4187-974b-39a9ad987908 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:56.480298 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2ae77845-a554-4187-974b-39a9ad987908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:56.488287 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 22:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 22:55:56.528674 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c107a6-dfb7-4f94-a16c-943e5c82c551 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:55:56.539250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c107a6-dfb7-4f94-a16c-943e5c82c551 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:55:56.542691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b157449-1af5-4ce2-a3c2-13fad6fb7b21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:55:56.543315 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b157449-1af5-4ce2-a3c2-13fad6fb7b21 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:56.543594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b157449-1af5-4ce2-a3c2-13fad6fb7b21 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:56.543867 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b157449-1af5-4ce2-a3c2-13fad6fb7b21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:56.552371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b157449-1af5-4ce2-a3c2-13fad6fb7b21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:55:56.553773 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b157449-1af5-4ce2-a3c2-13fad6fb7b21 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:55:56.563349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 728/1449] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:55:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:55:58.905776 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5923c-b655-4ac0-abc1-47f6bafce8f3 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:58.906408 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5923c-b655-4ac0-abc1-47f6bafce8f3 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:58.906408 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5923c-b655-4ac0-abc1-47f6bafce8f3 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:58.907717 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 22:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:55:58.913021 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cf697-ac83-4e0d-80f6-e76915683279 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:58.913459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cf697-ac83-4e0d-80f6-e76915683279 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:58.913564 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cf697-ac83-4e0d-80f6-e76915683279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:58.931623 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-6d4cf697-ac83-4e0d-80f6-e76915683279 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:55:58.932813 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6d4cf697-ac83-4e0d-80f6-e76915683279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:55:58.940575 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 22:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:55:58.978213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5923c-b655-4ac0-abc1-47f6bafce8f3 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:55:58.988899 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5923c-b655-4ac0-abc1-47f6bafce8f3 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:55:58.992401 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75ce0a-a0b8-4363-9ced-694feabf1c46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:55:58.993021 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75ce0a-a0b8-4363-9ced-694feabf1c46 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:55:58.993301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75ce0a-a0b8-4363-9ced-694feabf1c46 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:55:58.993576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75ce0a-a0b8-4363-9ced-694feabf1c46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:55:59.002219 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75ce0a-a0b8-4363-9ced-694feabf1c46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:55:59.003657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75ce0a-a0b8-4363-9ced-694feabf1c46 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:55:59.013735 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 723/1452] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:55:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 22:56:01.264079 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea3a76-7209-426d-a619-6cc7bdfdde52 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:01.264653 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea3a76-7209-426d-a619-6cc7bdfdde52 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:01.264653 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea3a76-7209-426d-a619-6cc7bdfdde52 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:01.265980 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 22:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:56:01.271380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23051d46-e116-4493-9901-281fb8efdeb8 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:01.271784 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23051d46-e116-4493-9901-281fb8efdeb8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:01.271964 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23051d46-e116-4493-9901-281fb8efdeb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:01.289612 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-23051d46-e116-4493-9901-281fb8efdeb8 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:56:01.290865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-23051d46-e116-4493-9901-281fb8efdeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:56:01.298536 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 22:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:56:01.336272 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea3a76-7209-426d-a619-6cc7bdfdde52 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:56:01.346668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea3a76-7209-426d-a619-6cc7bdfdde52 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:56:01.350053 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00112374-a1bd-4f7c-a41a-10b321ac5c32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:56:01.350697 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00112374-a1bd-4f7c-a41a-10b321ac5c32 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:01.350977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00112374-a1bd-4f7c-a41a-10b321ac5c32 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:01.351284 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00112374-a1bd-4f7c-a41a-10b321ac5c32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:01.360372 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00112374-a1bd-4f7c-a41a-10b321ac5c32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:56:01.361787 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00112374-a1bd-4f7c-a41a-10b321ac5c32 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:56:01.370852 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 731/1455] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:56:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:56:03.624267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0627d-88b0-4321-9aec-99db834d06f4 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:03.624830 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0627d-88b0-4321-9aec-99db834d06f4 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:03.624830 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0627d-88b0-4321-9aec-99db834d06f4 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:03.626247 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 22:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:56:03.631726 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d91131-72f1-49e6-8ddd-39239eee15a9 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:03.632197 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d91131-72f1-49e6-8ddd-39239eee15a9 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:03.632802 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d91131-72f1-49e6-8ddd-39239eee15a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:03.650652 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f2d91131-72f1-49e6-8ddd-39239eee15a9 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:56:03.651855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f2d91131-72f1-49e6-8ddd-39239eee15a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:56:03.659698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 22:56:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:56:03.696752 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0627d-88b0-4321-9aec-99db834d06f4 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:56:03.707049 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0627d-88b0-4321-9aec-99db834d06f4 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:56:03.711080 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84cfb11-a4d4-4c9d-bd6f-96ad55ca5c25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:56:03.711706 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84cfb11-a4d4-4c9d-bd6f-96ad55ca5c25 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:03.711978 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84cfb11-a4d4-4c9d-bd6f-96ad55ca5c25 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:03.712245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84cfb11-a4d4-4c9d-bd6f-96ad55ca5c25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:03.720749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84cfb11-a4d4-4c9d-bd6f-96ad55ca5c25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:56:03.722186 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84cfb11-a4d4-4c9d-bd6f-96ad55ca5c25 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:56:03.731160 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 726/1458] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:56:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:56:05.989073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d14963-1cd6-4144-8185-788082fef48a None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:05.989603 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d14963-1cd6-4144-8185-788082fef48a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:05.989603 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d14963-1cd6-4144-8185-788082fef48a None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:05.991002 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 22:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:56:05.997017 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ba21a-9d9d-46b8-bce5-5a8986c84e61 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:05.997485 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ba21a-9d9d-46b8-bce5-5a8986c84e61 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:05.997590 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ba21a-9d9d-46b8-bce5-5a8986c84e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:06.015003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-692ba21a-9d9d-46b8-bce5-5a8986c84e61 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:56:06.016185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-692ba21a-9d9d-46b8-bce5-5a8986c84e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:56:06.024440 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 22:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:56:06.061212 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d14963-1cd6-4144-8185-788082fef48a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:56:06.071798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d14963-1cd6-4144-8185-788082fef48a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:56:06.075267 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c061da7-ac93-45a7-987c-ffa3faba5ee2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:56:06.075895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c061da7-ac93-45a7-987c-ffa3faba5ee2 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:06.076177 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c061da7-ac93-45a7-987c-ffa3faba5ee2 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:06.076453 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c061da7-ac93-45a7-987c-ffa3faba5ee2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:06.085014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c061da7-ac93-45a7-987c-ffa3faba5ee2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:56:06.087157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c061da7-ac93-45a7-987c-ffa3faba5ee2 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:56:06.096225 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 734/1461] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:56:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:56:08.352505 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b50569-30cf-4e2c-8519-6322c5ce8826 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:08.353079 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b50569-30cf-4e2c-8519-6322c5ce8826 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:08.353079 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b50569-30cf-4e2c-8519-6322c5ce8826 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:08.354593 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 22:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:56:08.360239 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e2dbf-2bdb-4340-8637-1b4096f425e3 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:08.360668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e2dbf-2bdb-4340-8637-1b4096f425e3 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:08.360790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e2dbf-2bdb-4340-8637-1b4096f425e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:08.379075 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-624e2dbf-2bdb-4340-8637-1b4096f425e3 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:56:08.380249 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-624e2dbf-2bdb-4340-8637-1b4096f425e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:56:08.388937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 22:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 22:56:08.424224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b50569-30cf-4e2c-8519-6322c5ce8826 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:56:08.435078 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b50569-30cf-4e2c-8519-6322c5ce8826 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:56:08.438811 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c9f00-97e8-49c2-81ae-af5ac0b7e517 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:56:08.439442 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c9f00-97e8-49c2-81ae-af5ac0b7e517 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:08.439719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c9f00-97e8-49c2-81ae-af5ac0b7e517 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:08.439988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c9f00-97e8-49c2-81ae-af5ac0b7e517 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:08.449708 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c9f00-97e8-49c2-81ae-af5ac0b7e517 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:56:08.451478 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c9f00-97e8-49c2-81ae-af5ac0b7e517 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:56:08.462295 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 729/1464] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:56:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:56:10.690777 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3feb8e-2fe5-4f50-afeb-1429619b6280 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:10.691307 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3feb8e-2fe5-4f50-afeb-1429619b6280 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:10.691307 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3feb8e-2fe5-4f50-afeb-1429619b6280 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:10.692609 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 22:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:56:10.699056 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b5ef3-2ceb-4556-bf40-ede11650c753 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:10.699489 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b5ef3-2ceb-4556-bf40-ede11650c753 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:10.699615 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b5ef3-2ceb-4556-bf40-ede11650c753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:10.717932 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-464b5ef3-2ceb-4556-bf40-ede11650c753 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:56:10.719122 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-464b5ef3-2ceb-4556-bf40-ede11650c753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:56:10.727338 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 22:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:56:10.762980 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3feb8e-2fe5-4f50-afeb-1429619b6280 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:56:10.775063 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3feb8e-2fe5-4f50-afeb-1429619b6280 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:56:10.778553 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d725c5d-2e4e-4ff6-8a1e-f55cffd2eb16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:56:10.779164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d725c5d-2e4e-4ff6-8a1e-f55cffd2eb16 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:10.779438 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d725c5d-2e4e-4ff6-8a1e-f55cffd2eb16 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:10.779706 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d725c5d-2e4e-4ff6-8a1e-f55cffd2eb16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:10.788218 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d725c5d-2e4e-4ff6-8a1e-f55cffd2eb16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:56:10.790229 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d725c5d-2e4e-4ff6-8a1e-f55cffd2eb16 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:56:10.799164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 737/1467] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:56:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:56:13.004026 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b43171-d52c-4f6f-91be-d5548b7cbfa5 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:13.004569 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b43171-d52c-4f6f-91be-d5548b7cbfa5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:13.004569 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b43171-d52c-4f6f-91be-d5548b7cbfa5 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:13.005928 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 22:56:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:56:13.012013 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f4ae6-a98b-43fa-bb01-b5f206ca41bc None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:13.012445 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f4ae6-a98b-43fa-bb01-b5f206ca41bc None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:13.012559 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f4ae6-a98b-43fa-bb01-b5f206ca41bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:13.030056 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c05f4ae6-a98b-43fa-bb01-b5f206ca41bc None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:56:13.031260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c05f4ae6-a98b-43fa-bb01-b5f206ca41bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:56:13.038849 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 22:56:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:56:13.074516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b43171-d52c-4f6f-91be-d5548b7cbfa5 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:56:13.084857 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b43171-d52c-4f6f-91be-d5548b7cbfa5 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:56:13.088311 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6bf5f-c5ce-42f5-bd15-3ea484fce3c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:56:13.088930 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6bf5f-c5ce-42f5-bd15-3ea484fce3c1 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:13.089207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6bf5f-c5ce-42f5-bd15-3ea484fce3c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:13.089479 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6bf5f-c5ce-42f5-bd15-3ea484fce3c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:13.097920 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6bf5f-c5ce-42f5-bd15-3ea484fce3c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:56:13.099361 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6bf5f-c5ce-42f5-bd15-3ea484fce3c1 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:56:13.108535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 732/1470] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:56:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:56:46.280636 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86651a8b-634d-4767-afec-b8353a9cd755 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:46.281246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86651a8b-634d-4767-afec-b8353a9cd755 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:46.281246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86651a8b-634d-4767-afec-b8353a9cd755 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:46.282588 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 739/1471] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:56:46.287721 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6e3cb-fa3e-4b33-85a6-0353af387e11 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:46.288180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6e3cb-fa3e-4b33-85a6-0353af387e11 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:46.288292 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6e3cb-fa3e-4b33-85a6-0353af387e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:46.305358 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a8b6e3cb-fa3e-4b33-85a6-0353af387e11 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:56:46.306568 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a8b6e3cb-fa3e-4b33-85a6-0353af387e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:56:46.314164 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 733/1472] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 22:56:46.360886 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac201337-04ec-436c-9649-94ef4b371550 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:46.361182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac201337-04ec-436c-9649-94ef4b371550 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:46.361461 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac201337-04ec-436c-9649-94ef4b371550 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:46.362835 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 740/1473] 10.222.0.23 () {54 vars in 962 bytes} [Mon May 9 22:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:56:46.367528 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2997914-58eb-4e9e-a864-0ad522d4d7b1 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:46.367812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2997914-58eb-4e9e-a864-0ad522d4d7b1 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:46.368101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2997914-58eb-4e9e-a864-0ad522d4d7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:46.385818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-f2997914-58eb-4e9e-a864-0ad522d4d7b1 None None] MFA Rules not processed for user `a81caf0558f2449cbbc8f4cd9446f363`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:56:46.386996 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f2997914-58eb-4e9e-a864-0ad522d4d7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:56:46.393169 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 734/1474] 10.222.0.23 () {58 vars in 1059 bytes} [Mon May 9 22:56:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:56:46.401010 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac201337-04ec-436c-9649-94ef4b371550 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:56:46.411852 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac201337-04ec-436c-9649-94ef4b371550 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:56:46.415511 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7627d-8cc3-4972-a8bc-7bd48528df7d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81caf0558f2449cbbc8f4cd9446f363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:56:46.416144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7627d-8cc3-4972-a8bc-7bd48528df7d service glance] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:46.416421 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7627d-8cc3-4972-a8bc-7bd48528df7d service glance] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:46.416695 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7627d-8cc3-4972-a8bc-7bd48528df7d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:46.425875 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7627d-8cc3-4972-a8bc-7bd48528df7d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:56:46.427376 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7627d-8cc3-4972-a8bc-7bd48528df7d service glance] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:56:46.436528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 741/1475] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:56:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 22:56:48.757517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fb190-347c-402d-aaf3-35673286571f None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:48.758043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fb190-347c-402d-aaf3-35673286571f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:48.758043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fb190-347c-402d-aaf3-35673286571f None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:48.759386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 735/1476] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:56:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:56:48.764539 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0368af-519b-4e8b-9daa-84d47cc98e50 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:48.764986 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0368af-519b-4e8b-9daa-84d47cc98e50 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:48.765134 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0368af-519b-4e8b-9daa-84d47cc98e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:48.782916 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5a0368af-519b-4e8b-9daa-84d47cc98e50 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:56:48.784114 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5a0368af-519b-4e8b-9daa-84d47cc98e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:56:48.791758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 742/1477] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:56:48.814925 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fb190-347c-402d-aaf3-35673286571f None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:56:48.827453 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fb190-347c-402d-aaf3-35673286571f None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:56:48.831790 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904139e-27dd-4e4f-baa5-089dcfa344e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:56:48.832420 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904139e-27dd-4e4f-baa5-089dcfa344e6 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:48.832693 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904139e-27dd-4e4f-baa5-089dcfa344e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:48.832959 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904139e-27dd-4e4f-baa5-089dcfa344e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:48.833671 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b904139e-27dd-4e4f-baa5-089dcfa344e6 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:56:48.835439 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: WARNING py.warnings [None req-b904139e-27dd-4e4f-baa5-089dcfa344e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 22:56:48.835439 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: warnings.warn(msg) May 09 22:56:48.835439 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: May 09 22:56:48.836555 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b904139e-27dd-4e4f-baa5-089dcfa344e6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:56:48.854618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 736/1478] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 9 22:56:48 2022] GET /identity/v3/projects => generated 2553 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 22:56:51.079922 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fb2e5-9562-4b5e-a2f8-9fa4a6a2e56c None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:51.080508 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fb2e5-9562-4b5e-a2f8-9fa4a6a2e56c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:51.080508 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fb2e5-9562-4b5e-a2f8-9fa4a6a2e56c None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:51.081804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 743/1479] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:56:51.087467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a449d33-64d1-48f1-bbfb-c89af769858b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:51.087896 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a449d33-64d1-48f1-bbfb-c89af769858b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:51.088019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a449d33-64d1-48f1-bbfb-c89af769858b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:51.108022 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-8a449d33-64d1-48f1-bbfb-c89af769858b None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:56:51.109309 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8a449d33-64d1-48f1-bbfb-c89af769858b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:56:51.117208 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 737/1480] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:56:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:56:51.158465 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fb2e5-9562-4b5e-a2f8-9fa4a6a2e56c None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:56:51.173542 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fb2e5-9562-4b5e-a2f8-9fa4a6a2e56c None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:56:51.178916 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf5426-f2ea-4be4-9321-f62c6eb0ce84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:56:51.179545 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf5426-f2ea-4be4-9321-f62c6eb0ce84 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:51.179828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf5426-f2ea-4be4-9321-f62c6eb0ce84 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:51.180105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf5426-f2ea-4be4-9321-f62c6eb0ce84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:51.187991 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf5426-f2ea-4be4-9321-f62c6eb0ce84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:56:51.189554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf5426-f2ea-4be4-9321-f62c6eb0ce84 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:56:51.198738 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 744/1481] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:56:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 22:56:53.558198 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302fcfd-a67f-4b29-ad9c-246a4e0f207b None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:53.558743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302fcfd-a67f-4b29-ad9c-246a4e0f207b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:53.558743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302fcfd-a67f-4b29-ad9c-246a4e0f207b None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:53.560141 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 738/1482] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:56:53.565461 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fab52-3b05-424f-b11d-7a575c328d51 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:53.565886 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fab52-3b05-424f-b11d-7a575c328d51 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:53.566019 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fab52-3b05-424f-b11d-7a575c328d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:53.583631 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-181fab52-3b05-424f-b11d-7a575c328d51 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:56:53.584826 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-181fab52-3b05-424f-b11d-7a575c328d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:56:53.592541 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 745/1483] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:56:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:56:53.628160 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302fcfd-a67f-4b29-ad9c-246a4e0f207b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:56:53.639427 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302fcfd-a67f-4b29-ad9c-246a4e0f207b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:56:53.642893 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2157459a-5441-4b6c-97f2-c1d67ec46675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:56:53.643527 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157459a-5441-4b6c-97f2-c1d67ec46675 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:53.643807 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157459a-5441-4b6c-97f2-c1d67ec46675 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:53.644083 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157459a-5441-4b6c-97f2-c1d67ec46675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:53.652870 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2157459a-5441-4b6c-97f2-c1d67ec46675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:56:53.654331 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2157459a-5441-4b6c-97f2-c1d67ec46675 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:56:53.664026 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 739/1484] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:56:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:56:55.927050 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d423919-db14-43d7-87c2-9e980ffeee25 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:55.927590 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d423919-db14-43d7-87c2-9e980ffeee25 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:55.927590 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d423919-db14-43d7-87c2-9e980ffeee25 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:55.928876 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 746/1485] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:56:55.933980 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc244261-fafb-4a64-91d2-c9622963aad3 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:55.934429 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc244261-fafb-4a64-91d2-c9622963aad3 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:55.934566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc244261-fafb-4a64-91d2-c9622963aad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:55.952112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-cc244261-fafb-4a64-91d2-c9622963aad3 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:56:55.953280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-cc244261-fafb-4a64-91d2-c9622963aad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:56:55.961645 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 740/1486] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:56:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:56:55.997862 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d423919-db14-43d7-87c2-9e980ffeee25 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:56:56.008935 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d423919-db14-43d7-87c2-9e980ffeee25 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:56:56.012386 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9a636-815e-4a26-aca0-152c6e862903 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:56:56.013013 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9a636-815e-4a26-aca0-152c6e862903 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:56.013293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9a636-815e-4a26-aca0-152c6e862903 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:56.013564 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9a636-815e-4a26-aca0-152c6e862903 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:56.022966 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9a636-815e-4a26-aca0-152c6e862903 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:56:56.024381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9a636-815e-4a26-aca0-152c6e862903 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:56:56.033424 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 747/1487] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:56:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:56:58.370522 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f4930-1715-43db-a905-f84f87b1ffbe None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:58.371092 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f4930-1715-43db-a905-f84f87b1ffbe None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:58.371092 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f4930-1715-43db-a905-f84f87b1ffbe None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:58.372511 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 741/1488] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:56:58.377864 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b0556-e48c-4275-98a4-4eb7abd205a3 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:58.378367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b0556-e48c-4275-98a4-4eb7abd205a3 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:58.378474 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b0556-e48c-4275-98a4-4eb7abd205a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:58.397658 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-fb1b0556-e48c-4275-98a4-4eb7abd205a3 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:56:58.398856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-fb1b0556-e48c-4275-98a4-4eb7abd205a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:56:58.406410 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 748/1489] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:56:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:56:58.446159 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f4930-1715-43db-a905-f84f87b1ffbe None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:56:58.457806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f4930-1715-43db-a905-f84f87b1ffbe None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:56:58.461992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0157577c-a02a-4327-8a26-7e74f4567e6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:56:58.462657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157577c-a02a-4327-8a26-7e74f4567e6b service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:56:58.462936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157577c-a02a-4327-8a26-7e74f4567e6b service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:56:58.463213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157577c-a02a-4327-8a26-7e74f4567e6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:56:58.472263 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0157577c-a02a-4327-8a26-7e74f4567e6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:56:58.473681 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0157577c-a02a-4327-8a26-7e74f4567e6b service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:56:58.483038 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 742/1490] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:56:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:57:00.897266 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e395a-2537-498f-959c-c5a0fd16e4cc None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:57:00.897846 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e395a-2537-498f-959c-c5a0fd16e4cc None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:57:00.897846 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e395a-2537-498f-959c-c5a0fd16e4cc None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:57:00.899233 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 749/1491] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 9 22:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:57:00.904403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c81b21-b464-4818-aef7-973bc588d526 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:57:00.904813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c81b21-b464-4818-aef7-973bc588d526 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:57:00.905006 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c81b21-b464-4818-aef7-973bc588d526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:57:00.922013 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-16c81b21-b464-4818-aef7-973bc588d526 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:57:00.923211 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-16c81b21-b464-4818-aef7-973bc588d526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:57:00.931368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 743/1492] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 9 22:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:57:00.969168 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e395a-2537-498f-959c-c5a0fd16e4cc None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:57:00.979621 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e395a-2537-498f-959c-c5a0fd16e4cc None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:57:00.983347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d533a-5c2d-4464-aa65-28678365bfe6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:57:00.983970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d533a-5c2d-4464-aa65-28678365bfe6 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:57:00.984254 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d533a-5c2d-4464-aa65-28678365bfe6 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:57:00.984525 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d533a-5c2d-4464-aa65-28678365bfe6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:57:00.993476 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d533a-5c2d-4464-aa65-28678365bfe6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:57:00.994911 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d533a-5c2d-4464-aa65-28678365bfe6 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:57:01.004000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 750/1493] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:57:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:58:27.641605 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a0b5b-5905-41bf-9326-33b2cbb18426 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:58:27.642184 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a0b5b-5905-41bf-9326-33b2cbb18426 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:58:27.642184 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a0b5b-5905-41bf-9326-33b2cbb18426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:58:27.659917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-3d0a0b5b-5905-41bf-9326-33b2cbb18426 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:58:27.661097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3d0a0b5b-5905-41bf-9326-33b2cbb18426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:58:27.668866 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 744/1494] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 22:58:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:58:27.678134 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92874fbc-aedb-476e-b535-c6e74acbee15 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:58:27.678642 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92874fbc-aedb-476e-b535-c6e74acbee15 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:58:27.678741 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92874fbc-aedb-476e-b535-c6e74acbee15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:58:27.696216 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-92874fbc-aedb-476e-b535-c6e74acbee15 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:58:27.697376 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-92874fbc-aedb-476e-b535-c6e74acbee15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:58:27.704926 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 751/1495] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 22:58:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:58:27.709948 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a0b5b-5905-41bf-9326-33b2cbb18426 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:58:27.719383 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a0b5b-5905-41bf-9326-33b2cbb18426 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:58:27.723812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170ef7d-7141-4f7e-a181-e4af118d358f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:58:27.724449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170ef7d-7141-4f7e-a181-e4af118d358f admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:58:27.724725 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170ef7d-7141-4f7e-a181-e4af118d358f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:58:27.724997 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170ef7d-7141-4f7e-a181-e4af118d358f admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:58:27.725702 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170ef7d-7141-4f7e-a181-e4af118d358f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 22:58:27.726006 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170ef7d-7141-4f7e-a181-e4af118d358f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:58:27.727052 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: WARNING py.warnings [None req-b170ef7d-7141-4f7e-a181-e4af118d358f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 09 22:58:27.727052 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: warnings.warn(msg) May 09 22:58:27.727052 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: May 09 22:58:27.727923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170ef7d-7141-4f7e-a181-e4af118d358f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:58:27.739261 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 745/1496] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 22:58:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 22:58:27.746900 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92874fbc-aedb-476e-b535-c6e74acbee15 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:58:27.756058 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92874fbc-aedb-476e-b535-c6e74acbee15 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:58:27.760481 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb9228-8d3f-4cef-83ec-fc8ed328a49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:58:27.761110 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb9228-8d3f-4cef-83ec-fc8ed328a49a admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:58:27.761386 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb9228-8d3f-4cef-83ec-fc8ed328a49a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:58:27.761655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb9228-8d3f-4cef-83ec-fc8ed328a49a admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:58:27.762364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb9228-8d3f-4cef-83ec-fc8ed328a49a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:58:27.764104 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: WARNING py.warnings [None req-62fb9228-8d3f-4cef-83ec-fc8ed328a49a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 22:58:27.764104 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: warnings.warn(msg) May 09 22:58:27.764104 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: May 09 22:58:27.765264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb9228-8d3f-4cef-83ec-fc8ed328a49a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:58:27.798483 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 752/1497] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 22:58:27 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:58:27.803264 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170ef7d-7141-4f7e-a181-e4af118d358f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:58:27.813002 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170ef7d-7141-4f7e-a181-e4af118d358f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:58:27.867938 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f2530-b09f-405e-ba97-18c90fffd5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:58:27.868612 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f2530-b09f-405e-ba97-18c90fffd5a1 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:58:27.868888 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f2530-b09f-405e-ba97-18c90fffd5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:58:27.869159 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f2530-b09f-405e-ba97-18c90fffd5a1 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:58:27.869839 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f2530-b09f-405e-ba97-18c90fffd5a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:58:27.870885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: WARNING py.warnings [None req-b08f2530-b09f-405e-ba97-18c90fffd5a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 22:58:27.870885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: warnings.warn(msg) May 09 22:58:27.870885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: May 09 22:58:27.872071 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f2530-b09f-405e-ba97-18c90fffd5a1 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:58:27.891382 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 746/1498] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 22:58:27 2022] POST /identity/v3/users => generated 288 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 22:58:27.895250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb9228-8d3f-4cef-83ec-fc8ed328a49a admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:58:27.905130 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb9228-8d3f-4cef-83ec-fc8ed328a49a admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:58:27.909664 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee9947-b7fa-4979-99b9-08a52e17529c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:58:27.910342 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee9947-b7fa-4979-99b9-08a52e17529c admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:58:27.910628 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee9947-b7fa-4979-99b9-08a52e17529c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:58:27.910905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee9947-b7fa-4979-99b9-08a52e17529c admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:58:27.911621 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee9947-b7fa-4979-99b9-08a52e17529c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:58:27.912625 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: WARNING py.warnings [None req-6dee9947-b7fa-4979-99b9-08a52e17529c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 09 22:58:27.912625 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: warnings.warn(msg) May 09 22:58:27.912625 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: May 09 22:58:27.913496 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee9947-b7fa-4979-99b9-08a52e17529c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:58:27.926366 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 753/1499] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 22:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 22:58:27.930331 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f2530-b09f-405e-ba97-18c90fffd5a1 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:58:27.940016 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f2530-b09f-405e-ba97-18c90fffd5a1 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:58:27.945210 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6decfd-9062-4234-9943-385768bb25e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:58:27.945886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6decfd-9062-4234-9943-385768bb25e6 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:58:27.946201 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6decfd-9062-4234-9943-385768bb25e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:58:27.946475 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6decfd-9062-4234-9943-385768bb25e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:58:27.947186 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6decfd-9062-4234-9943-385768bb25e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:58:27.948211 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: WARNING py.warnings [None req-3f6decfd-9062-4234-9943-385768bb25e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 09 22:58:27.948211 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: warnings.warn(msg) May 09 22:58:27.948211 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: May 09 22:58:27.949096 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6decfd-9062-4234-9943-385768bb25e6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:58:27.962227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 747/1500] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 22:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 22:58:27.966248 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee9947-b7fa-4979-99b9-08a52e17529c admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:58:27.976029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee9947-b7fa-4979-99b9-08a52e17529c admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:58:27.980465 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ab525-59c4-4663-9cd7-7db3f72b112c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:58:27.981324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ab525-59c4-4663-9cd7-7db3f72b112c admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:58:27.981600 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ab525-59c4-4663-9cd7-7db3f72b112c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:58:27.981873 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ab525-59c4-4663-9cd7-7db3f72b112c admin admin] PATH_INFO: `/v3/projects/757c0b2ca1414054900dd58d856b5a9f/users/9b923818f1b64c988cbd3caf8c9d788a/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:58:27.996768 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ab525-59c4-4663-9cd7-7db3f72b112c admin admin] RBAC: Authorizing `identity:create_grant(project_id=757c0b2ca1414054900dd58d856b5a9f, user_id=9b923818f1b64c988cbd3caf8c9d788a, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:58:27.997787 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: WARNING py.warnings [None req-0d9ab525-59c4-4663-9cd7-7db3f72b112c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 22:58:27.997787 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: warnings.warn(msg) May 09 22:58:27.997787 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: May 09 22:58:27.999720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ab525-59c4-4663-9cd7-7db3f72b112c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:58:28.012762 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 754/1501] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 22:58:27 2022] PUT /identity/v3/projects/757c0b2ca1414054900dd58d856b5a9f/users/9b923818f1b64c988cbd3caf8c9d788a/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 22:58:28.023762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950afb82-3ecd-436b-9501-41616fa6aff4 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:58:28.024056 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950afb82-3ecd-436b-9501-41616fa6aff4 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:58:28.024336 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950afb82-3ecd-436b-9501-41616fa6aff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:58:28.049206 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-950afb82-3ecd-436b-9501-41616fa6aff4 None None] MFA Rules not processed for user `9b923818f1b64c988cbd3caf8c9d788a`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:58:28.050944 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-950afb82-3ecd-436b-9501-41616fa6aff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:58:28.110559 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 748/1502] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 22:58:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 22:58:28.123919 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986208c-fec4-4f89-a035-34a8dfb92730 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:58:28.124237 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986208c-fec4-4f89-a035-34a8dfb92730 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:58:28.124515 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986208c-fec4-4f89-a035-34a8dfb92730 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:58:28.125866 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 755/1503] 10.222.0.23 () {54 vars in 963 bytes} [Mon May 9 22:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:58:28.131354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4e177-8a58-4afd-b683-d8c30f36d85b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:58:28.131663 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4e177-8a58-4afd-b683-d8c30f36d85b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:58:28.131939 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4e177-8a58-4afd-b683-d8c30f36d85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:58:28.150981 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-24c4e177-8a58-4afd-b683-d8c30f36d85b None None] MFA Rules not processed for user `5fe668a2cee040c7a56d1f8685744cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:58:28.152265 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-24c4e177-8a58-4afd-b683-d8c30f36d85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:58:28.200971 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 749/1504] 10.222.0.23 () {58 vars in 1060 bytes} [Mon May 9 22:58:28 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 22:58:28.208367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986208c-fec4-4f89-a035-34a8dfb92730 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:58:28.218719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986208c-fec4-4f89-a035-34a8dfb92730 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:58:28.222691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea6019-e73b-473b-b6d3-cb3baec52d5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:58:28.223317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea6019-e73b-473b-b6d3-cb3baec52d5a service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:58:28.223597 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea6019-e73b-473b-b6d3-cb3baec52d5a service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:58:28.223872 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea6019-e73b-473b-b6d3-cb3baec52d5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:58:28.231820 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea6019-e73b-473b-b6d3-cb3baec52d5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:58:28.233496 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea6019-e73b-473b-b6d3-cb3baec52d5a service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:58:28.242647 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 756/1505] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:58:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 22:58:28.287811 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff698f36-ffd8-478f-983d-a947014dbd28 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:58:28.288104 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff698f36-ffd8-478f-983d-a947014dbd28 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:58:28.288377 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff698f36-ffd8-478f-983d-a947014dbd28 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:58:28.289669 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 750/1506] 10.222.0.23 () {52 vars in 829 bytes} [Mon May 9 22:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 22:58:28.293254 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea6019-e73b-473b-b6d3-cb3baec52d5a service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:58:28.352167 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea6019-e73b-473b-b6d3-cb3baec52d5a service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:58:28.356624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31986a6-671a-45f8-b12c-78a8b0f70e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:58:28.357472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31986a6-671a-45f8-b12c-78a8b0f70e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:58:28.358010 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31986a6-671a-45f8-b12c-78a8b0f70e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:58:28.358311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31986a6-671a-45f8-b12c-78a8b0f70e44 admin admin] PATH_INFO: `/v3/users/9b923818f1b64c988cbd3caf8c9d788a` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:58:28.359474 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31986a6-671a-45f8-b12c-78a8b0f70e44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b923818f1b64c988cbd3caf8c9d788a)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:58:28.360484 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: WARNING py.warnings [None req-a31986a6-671a-45f8-b12c-78a8b0f70e44 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 22:58:28.360484 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: warnings.warn(msg) May 09 22:58:28.360484 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: May 09 22:58:28.361691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31986a6-671a-45f8-b12c-78a8b0f70e44 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:58:28.423916 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a31986a6-671a-45f8-b12c-78a8b0f70e44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b923818f1b64c988cbd3caf8c9d788a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 22:58:28.436434 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a31986a6-671a-45f8-b12c-78a8b0f70e44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b923818f1b64c988cbd3caf8c9d788a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 22:58:28.446226 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a31986a6-671a-45f8-b12c-78a8b0f70e44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b923818f1b64c988cbd3caf8c9d788a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 22:58:28.446935 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a31986a6-671a-45f8-b12c-78a8b0f70e44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b923818f1b64c988cbd3caf8c9d788a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 22:58:28.467346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a31986a6-671a-45f8-b12c-78a8b0f70e44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b923818f1b64c988cbd3caf8c9d788a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 22:58:28.474045 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 757/1507] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 22:58:28 2022] DELETE /identity/v3/users/9b923818f1b64c988cbd3caf8c9d788a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 22:58:28.478465 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423765bc-b0e2-4c57-a272-71e89402db16 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:58:28.478755 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423765bc-b0e2-4c57-a272-71e89402db16 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:58:28.479032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423765bc-b0e2-4c57-a272-71e89402db16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:58:28.497113 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-423765bc-b0e2-4c57-a272-71e89402db16 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 22:58:28.498334 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-423765bc-b0e2-4c57-a272-71e89402db16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:58:28.553445 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 751/1508] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 22:58:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 22:58:28.563391 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31986a6-671a-45f8-b12c-78a8b0f70e44 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:58:28.564770 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a31986a6-671a-45f8-b12c-78a8b0f70e44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:58:28.619354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31986a6-671a-45f8-b12c-78a8b0f70e44 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:58:28.623013 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cdea6c-9870-4835-843d-0244d7d20bff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:58:28.623655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdea6c-9870-4835-843d-0244d7d20bff service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:58:28.623933 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdea6c-9870-4835-843d-0244d7d20bff service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:58:28.624206 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdea6c-9870-4835-843d-0244d7d20bff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:58:28.633102 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cdea6c-9870-4835-843d-0244d7d20bff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:58:28.634579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cdea6c-9870-4835-843d-0244d7d20bff service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:58:28.644299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 758/1509] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 22:58:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 22:58:28.962605 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423765bc-b0e2-4c57-a272-71e89402db16 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 22:58:28.964073 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-423765bc-b0e2-4c57-a272-71e89402db16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 22:58:28.975487 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423765bc-b0e2-4c57-a272-71e89402db16 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 22:58:28.980088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07533e12-5e33-48b6-983a-c2aaca3b5d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 22:58:28.981152 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07533e12-5e33-48b6-983a-c2aaca3b5d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 22:58:28.981277 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07533e12-5e33-48b6-983a-c2aaca3b5d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 22:58:28.981486 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07533e12-5e33-48b6-983a-c2aaca3b5d62 admin admin] PATH_INFO: `/v3/projects/757c0b2ca1414054900dd58d856b5a9f` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 22:58:28.983100 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07533e12-5e33-48b6-983a-c2aaca3b5d62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=757c0b2ca1414054900dd58d856b5a9f)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 22:58:28.984179 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: WARNING py.warnings [None req-07533e12-5e33-48b6-983a-c2aaca3b5d62 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 22:58:28.984179 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: warnings.warn(msg) May 09 22:58:28.984179 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: May 09 22:58:28.985374 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07533e12-5e33-48b6-983a-c2aaca3b5d62 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 22:58:29.043780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-07533e12-5e33-48b6-983a-c2aaca3b5d62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '757c0b2ca1414054900dd58d856b5a9f', 'request_id': 'req-07533e12-5e33-48b6-983a-c2aaca3b5d62'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 22:58:29.056812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-07533e12-5e33-48b6-983a-c2aaca3b5d62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '757c0b2ca1414054900dd58d856b5a9f', 'request_id': 'req-07533e12-5e33-48b6-983a-c2aaca3b5d62'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 22:58:29.062795 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-07533e12-5e33-48b6-983a-c2aaca3b5d62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '757c0b2ca1414054900dd58d856b5a9f', 'request_id': 'req-07533e12-5e33-48b6-983a-c2aaca3b5d62'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 22:58:29.077197 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-07533e12-5e33-48b6-983a-c2aaca3b5d62 admin admin] The token cache is being invalidate because project 757c0b2ca1414054900dd58d856b5a9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 22:58:29.077481 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-07533e12-5e33-48b6-983a-c2aaca3b5d62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 22:58:29.078099 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-07533e12-5e33-48b6-983a-c2aaca3b5d62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 22:58:29.079944 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 752/1510] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 22:58:28 2022] DELETE /identity/v3/projects/757c0b2ca1414054900dd58d856b5a9f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:00:11.663928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428d2df-260f-4d09-a661-a6d5f7ae94c8 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:11.664472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428d2df-260f-4d09-a661-a6d5f7ae94c8 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:11.664472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428d2df-260f-4d09-a661-a6d5f7ae94c8 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:11.665865 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 23:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 23:00:11.670602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee82c2-1a2a-4a8a-9153-5890a2c28294 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:11.671068 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee82c2-1a2a-4a8a-9153-5890a2c28294 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:11.671176 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee82c2-1a2a-4a8a-9153-5890a2c28294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:11.692252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-21ee82c2-1a2a-4a8a-9153-5890a2c28294 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:00:11.693504 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-21ee82c2-1a2a-4a8a-9153-5890a2c28294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:00:11.750719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 23:00:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:00:11.920092 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428d2df-260f-4d09-a661-a6d5f7ae94c8 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:00:11.921688 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1428d2df-260f-4d09-a661-a6d5f7ae94c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:00:11.995549 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428d2df-260f-4d09-a661-a6d5f7ae94c8 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:00:12.000868 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c5144-1250-44fa-9bc0-d63adba59771 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:00:12.002995 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c5144-1250-44fa-9bc0-d63adba59771 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:12.003275 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c5144-1250-44fa-9bc0-d63adba59771 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:12.003545 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c5144-1250-44fa-9bc0-d63adba59771 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:12.011688 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c5144-1250-44fa-9bc0-d63adba59771 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:00:12.013302 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c5144-1250-44fa-9bc0-d63adba59771 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:00:12.022907 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 760/1513] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:00:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:00:12.601053 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9cbca-82cb-4a5c-9d0d-d295e8049865 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:12.601342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9cbca-82cb-4a5c-9d0d-d295e8049865 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:12.601613 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9cbca-82cb-4a5c-9d0d-d295e8049865 None None] PATH_INFO: `/v3` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:12.602944 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 754/1514] 10.222.0.55 () {54 vars in 925 bytes} [Mon May 9 23:00:12 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:00:12.609532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9accb76-afde-45f4-95d6-825f38735d32 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:12.609828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9accb76-afde-45f4-95d6-825f38735d32 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:12.610131 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9accb76-afde-45f4-95d6-825f38735d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:12.628597 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c9accb76-afde-45f4-95d6-825f38735d32 None None] MFA Rules not processed for user `cf9bc2fa959d4706b9f9e87509829059`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:00:12.629808 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c9accb76-afde-45f4-95d6-825f38735d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:00:12.691312 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 761/1515] 10.222.0.55 () {58 vars in 1015 bytes} [Mon May 9 23:00:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:00:12.930035 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee7a81-cde8-43c5-89c1-169524dae805 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:12.930610 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee7a81-cde8-43c5-89c1-169524dae805 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:12.930610 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee7a81-cde8-43c5-89c1-169524dae805 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:12.931949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 755/1516] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 23:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 23:00:12.936792 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841454c-4d5f-43e4-94d5-8d4727731a2d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:12.937086 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841454c-4d5f-43e4-94d5-8d4727731a2d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:12.937366 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841454c-4d5f-43e4-94d5-8d4727731a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:12.955475 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-8841454c-4d5f-43e4-94d5-8d4727731a2d None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:00:12.956967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-8841454c-4d5f-43e4-94d5-8d4727731a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:00:12.964737 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 762/1517] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 23:00:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:00:13.122586 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee7a81-cde8-43c5-89c1-169524dae805 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:00:13.136764 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee7a81-cde8-43c5-89c1-169524dae805 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:00:13.142060 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9ed3d-56fe-45d3-9472-d88a298d85d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:00:13.142713 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9ed3d-56fe-45d3-9472-d88a298d85d6 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:13.142988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9ed3d-56fe-45d3-9472-d88a298d85d6 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:13.143257 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9ed3d-56fe-45d3-9472-d88a298d85d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:13.151004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9ed3d-56fe-45d3-9472-d88a298d85d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:00:13.152600 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9ed3d-56fe-45d3-9472-d88a298d85d6 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:00:13.162481 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 756/1518] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:00:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:00:16.566997 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08198c14-e8d4-415f-8101-047ee7934381 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:16.567503 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08198c14-e8d4-415f-8101-047ee7934381 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:16.567619 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08198c14-e8d4-415f-8101-047ee7934381 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:16.569079 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 763/1519] 10.222.0.23 () {54 vars in 960 bytes} [Mon May 9 23:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 23:00:16.574344 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e893d-44e1-4f3c-a2a9-805bde631173 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:16.574824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e893d-44e1-4f3c-a2a9-805bde631173 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:16.575052 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e893d-44e1-4f3c-a2a9-805bde631173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:16.593095 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-286e893d-44e1-4f3c-a2a9-805bde631173 None None] MFA Rules not processed for user `cf9bc2fa959d4706b9f9e87509829059`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:00:16.594323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-286e893d-44e1-4f3c-a2a9-805bde631173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:00:16.603753 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 757/1520] 10.222.0.23 () {58 vars in 1057 bytes} [Mon May 9 23:00:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:00:16.612846 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08198c14-e8d4-415f-8101-047ee7934381 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:00:16.626279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08198c14-e8d4-415f-8101-047ee7934381 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:00:16.631174 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f8d8d-29d8-45c7-9f9b-d6b69eae66c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf9bc2fa959d4706b9f9e87509829059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:00:16.631804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f8d8d-29d8-45c7-9f9b-d6b69eae66c2 service placement] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:16.632115 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f8d8d-29d8-45c7-9f9b-d6b69eae66c2 service placement] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:16.632389 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f8d8d-29d8-45c7-9f9b-d6b69eae66c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:16.640863 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970f8d8d-29d8-45c7-9f9b-d6b69eae66c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:00:16.642986 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970f8d8d-29d8-45c7-9f9b-d6b69eae66c2 service placement] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:00:16.652403 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 764/1521] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:00:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:00:24.474343 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d38dc-b53f-4de9-ada4-99f0f66b4457 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:24.474903 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d38dc-b53f-4de9-ada4-99f0f66b4457 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:24.474903 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d38dc-b53f-4de9-ada4-99f0f66b4457 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:24.476235 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 23:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 23:00:24.480957 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fffd7-1307-4f88-8ebe-3a2cc780a0c8 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:24.481366 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fffd7-1307-4f88-8ebe-3a2cc780a0c8 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:24.481512 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fffd7-1307-4f88-8ebe-3a2cc780a0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:24.499698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-a33fffd7-1307-4f88-8ebe-3a2cc780a0c8 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:00:24.502298 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a33fffd7-1307-4f88-8ebe-3a2cc780a0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:00:24.512485 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 23:00:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:00:24.663276 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d38dc-b53f-4de9-ada4-99f0f66b4457 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:00:24.678128 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d38dc-b53f-4de9-ada4-99f0f66b4457 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:00:24.684074 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e824f-07f2-44a7-8c11-1cfc1877ae41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:00:24.684713 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e824f-07f2-44a7-8c11-1cfc1877ae41 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:24.684993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e824f-07f2-44a7-8c11-1cfc1877ae41 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:24.685266 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e824f-07f2-44a7-8c11-1cfc1877ae41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:24.695167 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e824f-07f2-44a7-8c11-1cfc1877ae41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:00:24.696948 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e824f-07f2-44a7-8c11-1cfc1877ae41 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:00:24.706664 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 759/1524] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:00:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:00:25.598090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118cdea-701a-42ff-917e-15790fd4334c None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:25.598718 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118cdea-701a-42ff-917e-15790fd4334c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:25.598718 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118cdea-701a-42ff-917e-15790fd4334c None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:25.600067 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 23:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 23:00:25.604925 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2918d9-9e5c-4fff-b390-df9a3889e301 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:25.605336 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2918d9-9e5c-4fff-b390-df9a3889e301 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:25.605511 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2918d9-9e5c-4fff-b390-df9a3889e301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:25.623251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-8f2918d9-9e5c-4fff-b390-df9a3889e301 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:00:25.624444 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8f2918d9-9e5c-4fff-b390-df9a3889e301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:00:25.632500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 23:00:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:00:25.785320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118cdea-701a-42ff-917e-15790fd4334c None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:00:25.799515 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118cdea-701a-42ff-917e-15790fd4334c None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:00:25.804895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eed7d3-676e-4c85-ba32-4f469bf3ec9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:00:25.805524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eed7d3-676e-4c85-ba32-4f469bf3ec9d service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:25.805805 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eed7d3-676e-4c85-ba32-4f469bf3ec9d service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:25.806083 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eed7d3-676e-4c85-ba32-4f469bf3ec9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:25.814159 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eed7d3-676e-4c85-ba32-4f469bf3ec9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:00:25.815730 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eed7d3-676e-4c85-ba32-4f469bf3ec9d service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:00:25.824935 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 767/1527] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:00:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:00:30.002477 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad6684-a51f-4ccb-a883-1ebe109cf342 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:30.002993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad6684-a51f-4ccb-a883-1ebe109cf342 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:30.002993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad6684-a51f-4ccb-a883-1ebe109cf342 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:30.004336 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 23:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 23:00:30.009717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448440a-0a89-4357-ad1e-39007f054351 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:30.010189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448440a-0a89-4357-ad1e-39007f054351 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:30.010309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448440a-0a89-4357-ad1e-39007f054351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:30.028155 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-0448440a-0a89-4357-ad1e-39007f054351 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:00:30.029346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0448440a-0a89-4357-ad1e-39007f054351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:00:30.037584 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 23:00:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:00:30.073139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad6684-a51f-4ccb-a883-1ebe109cf342 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:00:30.074851 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-15ad6684-a51f-4ccb-a883-1ebe109cf342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:00:30.128413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad6684-a51f-4ccb-a883-1ebe109cf342 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:00:30.131869 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7a978-bf7a-4bf9-9ac9-9549afdf10b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:00:30.132497 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7a978-bf7a-4bf9-9ac9-9549afdf10b4 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:30.132770 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7a978-bf7a-4bf9-9ac9-9549afdf10b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:30.133038 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7a978-bf7a-4bf9-9ac9-9549afdf10b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:30.142576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7a978-bf7a-4bf9-9ac9-9549afdf10b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:00:30.143999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7a978-bf7a-4bf9-9ac9-9549afdf10b4 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:00:30.153083 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 762/1530] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:00:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:00:32.347905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bc4b8-07d4-4cfe-9388-9e92a10b39e0 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:32.348502 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bc4b8-07d4-4cfe-9388-9e92a10b39e0 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:32.348502 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bc4b8-07d4-4cfe-9388-9e92a10b39e0 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:32.349805 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 23:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 23:00:32.354946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b83b2-b706-4fbb-aa3a-b3a8cf752ec1 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:32.355402 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b83b2-b706-4fbb-aa3a-b3a8cf752ec1 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:32.355538 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b83b2-b706-4fbb-aa3a-b3a8cf752ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:32.374755 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-840b83b2-b706-4fbb-aa3a-b3a8cf752ec1 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:00:32.375931 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-840b83b2-b706-4fbb-aa3a-b3a8cf752ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:00:32.383667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 23:00:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:00:32.418431 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6bc4b8-07d4-4cfe-9388-9e92a10b39e0 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:00:32.428813 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6bc4b8-07d4-4cfe-9388-9e92a10b39e0 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:00:32.432239 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287e5869-59b1-486f-9c9c-1fe64e8cd8c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:00:32.432861 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e5869-59b1-486f-9c9c-1fe64e8cd8c6 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:32.433138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e5869-59b1-486f-9c9c-1fe64e8cd8c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:32.433409 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e5869-59b1-486f-9c9c-1fe64e8cd8c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:32.441861 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287e5869-59b1-486f-9c9c-1fe64e8cd8c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:00:32.443299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287e5869-59b1-486f-9c9c-1fe64e8cd8c6 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:00:32.452229 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 770/1533] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:00:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:00:36.098959 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85b358-e420-4204-9eb5-7ed4507a17f5 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:36.099563 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85b358-e420-4204-9eb5-7ed4507a17f5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:36.099563 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85b358-e420-4204-9eb5-7ed4507a17f5 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:36.100914 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 23:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 23:00:36.106430 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf9269-6d80-4faa-bc18-90985ca11edd None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:36.106871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf9269-6d80-4faa-bc18-90985ca11edd None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:36.106981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf9269-6d80-4faa-bc18-90985ca11edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:36.126246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-febf9269-6d80-4faa-bc18-90985ca11edd None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:00:36.127420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-febf9269-6d80-4faa-bc18-90985ca11edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:00:36.135019 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 23:00:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:00:36.414759 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85b358-e420-4204-9eb5-7ed4507a17f5 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:00:36.416382 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-fb85b358-e420-4204-9eb5-7ed4507a17f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:00:36.470257 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85b358-e420-4204-9eb5-7ed4507a17f5 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:00:36.473690 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ebdc6-f1cc-45af-93e7-0ecda2afb1b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:00:36.474343 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ebdc6-f1cc-45af-93e7-0ecda2afb1b1 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:36.474618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ebdc6-f1cc-45af-93e7-0ecda2afb1b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:36.474882 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ebdc6-f1cc-45af-93e7-0ecda2afb1b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:36.483766 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ebdc6-f1cc-45af-93e7-0ecda2afb1b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:00:36.485786 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ebdc6-f1cc-45af-93e7-0ecda2afb1b1 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:00:36.494816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 765/1536] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:00:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:00:38.809078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb61f9-99e5-4839-b40d-281e86e3cd0c None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:38.809617 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb61f9-99e5-4839-b40d-281e86e3cd0c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:38.809617 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb61f9-99e5-4839-b40d-281e86e3cd0c None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:38.811011 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 23:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 23:00:38.816362 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e51d7f-456d-4cff-854f-3c1646e8302e None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:38.816797 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e51d7f-456d-4cff-854f-3c1646e8302e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:38.816923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e51d7f-456d-4cff-854f-3c1646e8302e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:38.834336 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-f0e51d7f-456d-4cff-854f-3c1646e8302e None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:00:38.835545 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f0e51d7f-456d-4cff-854f-3c1646e8302e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:00:38.843541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 23:00:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:00:39.116840 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb61f9-99e5-4839-b40d-281e86e3cd0c None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:00:39.118603 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ecfb61f9-99e5-4839-b40d-281e86e3cd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:00:39.131512 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb61f9-99e5-4839-b40d-281e86e3cd0c None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:00:39.134992 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809134a-bc66-42df-bc17-41149fd7d978 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:00:39.135631 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809134a-bc66-42df-bc17-41149fd7d978 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:00:39.135914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809134a-bc66-42df-bc17-41149fd7d978 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:00:39.136189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809134a-bc66-42df-bc17-41149fd7d978 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:00:39.145289 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809134a-bc66-42df-bc17-41149fd7d978 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:00:39.147227 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809134a-bc66-42df-bc17-41149fd7d978 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:00:39.156478 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 773/1539] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:00:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:11.352048 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc737d-a0dc-4884-963f-b78bc86c08f1 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:11.353370 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc737d-a0dc-4884-963f-b78bc86c08f1 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:11.353829 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc737d-a0dc-4884-963f-b78bc86c08f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:11.386812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-7bbc737d-a0dc-4884-963f-b78bc86c08f1 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:11.388564 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7bbc737d-a0dc-4884-963f-b78bc86c08f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:11.400681 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 767/1540] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:02:11.413369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c13121-33cb-4f0f-a798-95c07fb34e94 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:11.414527 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c13121-33cb-4f0f-a798-95c07fb34e94 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:11.415010 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c13121-33cb-4f0f-a798-95c07fb34e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:11.437841 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b400f-9438-44aa-a6e3-ab38f00e562f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:11.439078 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b400f-9438-44aa-a6e3-ab38f00e562f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:11.439694 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b400f-9438-44aa-a6e3-ab38f00e562f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:11.455722 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-43c13121-33cb-4f0f-a798-95c07fb34e94 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:11.460207 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-43c13121-33cb-4f0f-a798-95c07fb34e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:11.477793 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-9b0b400f-9438-44aa-a6e3-ab38f00e562f None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:11.479208 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9b0b400f-9438-44aa-a6e3-ab38f00e562f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:11.481153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 774/1541] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:02:11.486894 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c13121-33cb-4f0f-a798-95c07fb34e94 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:11.496518 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 768/1542] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:11.500421 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c13121-33cb-4f0f-a798-95c07fb34e94 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:11.506444 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905d956-6454-4402-bc22-c23ea8597006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:11.506630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266234e8-b4f0-48e3-8960-86d0f81df71e None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:11.506890 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266234e8-b4f0-48e3-8960-86d0f81df71e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:11.507169 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266234e8-b4f0-48e3-8960-86d0f81df71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:11.507961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905d956-6454-4402-bc22-c23ea8597006 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:11.508456 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905d956-6454-4402-bc22-c23ea8597006 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:11.508949 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905d956-6454-4402-bc22-c23ea8597006 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:11.509877 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905d956-6454-4402-bc22-c23ea8597006 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:02:11.514491 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905d956-6454-4402-bc22-c23ea8597006 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:11.516554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: WARNING py.warnings [None req-5905d956-6454-4402-bc22-c23ea8597006 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 09 23:02:11.516554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: warnings.warn(msg) May 09 23:02:11.516554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: May 09 23:02:11.518554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905d956-6454-4402-bc22-c23ea8597006 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:11.536465 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-266234e8-b4f0-48e3-8960-86d0f81df71e None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:11.537716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-266234e8-b4f0-48e3-8960-86d0f81df71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:11.539932 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 775/1543] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 23:02:11.541439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f980263-2943-42ff-9b1d-b808b4ef92f1 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:11.541965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f980263-2943-42ff-9b1d-b808b4ef92f1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:11.542471 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f980263-2943-42ff-9b1d-b808b4ef92f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:11.550239 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 769/1544] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:02:11.557573 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266234e8-b4f0-48e3-8960-86d0f81df71e None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:11.571391 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-6f980263-2943-42ff-9b1d-b808b4ef92f1 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:11.571780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266234e8-b4f0-48e3-8960-86d0f81df71e None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:11.573203 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6f980263-2943-42ff-9b1d-b808b4ef92f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:11.586689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 776/1545] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:11.588286 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f980263-2943-42ff-9b1d-b808b4ef92f1 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:11.595582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b3e4c6-a4f2-479f-980f-5b1f3bc37987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:11.596277 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3e4c6-a4f2-479f-980f-5b1f3bc37987 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:11.596555 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3e4c6-a4f2-479f-980f-5b1f3bc37987 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:11.596830 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3e4c6-a4f2-479f-980f-5b1f3bc37987 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:11.597569 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b3e4c6-a4f2-479f-980f-5b1f3bc37987 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:02:11.597874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b3e4c6-a4f2-479f-980f-5b1f3bc37987 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:11.599587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b3e4c6-a4f2-479f-980f-5b1f3bc37987 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:11.608265 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f980263-2943-42ff-9b1d-b808b4ef92f1 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:11.618357 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bfcbb6-2705-456a-8937-3f1b472e2907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:11.619735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfcbb6-2705-456a-8937-3f1b472e2907 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:11.620247 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfcbb6-2705-456a-8937-3f1b472e2907 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:11.620755 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfcbb6-2705-456a-8937-3f1b472e2907 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:11.621682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bfcbb6-2705-456a-8937-3f1b472e2907 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:11.623030 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 770/1546] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 23:02:11.624232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bfcbb6-2705-456a-8937-3f1b472e2907 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:11.624434 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03925c7f-3986-44b1-809b-008689a8f20b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:11.624700 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03925c7f-3986-44b1-809b-008689a8f20b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:11.624982 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03925c7f-3986-44b1-809b-008689a8f20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:11.666547 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-03925c7f-3986-44b1-809b-008689a8f20b None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:11.667747 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-03925c7f-3986-44b1-809b-008689a8f20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:11.689543 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 771/1547] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:11.690415 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03925c7f-3986-44b1-809b-008689a8f20b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:11.694292 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 777/1548] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:11 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:11.695895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28797c32-117d-4503-a6fa-f2acf037c00d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:11.696288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28797c32-117d-4503-a6fa-f2acf037c00d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:11.696575 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28797c32-117d-4503-a6fa-f2acf037c00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:11.704692 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03925c7f-3986-44b1-809b-008689a8f20b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:11.731307 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-28797c32-117d-4503-a6fa-f2acf037c00d None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:11.732835 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-28797c32-117d-4503-a6fa-f2acf037c00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:11.795561 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737a3cb-cc0a-4d99-a890-3100fb9c9fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:11.797525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737a3cb-cc0a-4d99-a890-3100fb9c9fb6 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:11.797525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737a3cb-cc0a-4d99-a890-3100fb9c9fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:11.797525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737a3cb-cc0a-4d99-a890-3100fb9c9fb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:11.797525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2737a3cb-cc0a-4d99-a890-3100fb9c9fb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:11.798500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: WARNING py.warnings [None req-2737a3cb-cc0a-4d99-a890-3100fb9c9fb6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 23:02:11.798500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: warnings.warn(msg) May 09 23:02:11.798500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: May 09 23:02:11.799752 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2737a3cb-cc0a-4d99-a890-3100fb9c9fb6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:11.826525 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 778/1549] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:11.827544 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28797c32-117d-4503-a6fa-f2acf037c00d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:11.828785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 772/1550] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:11 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:11.830233 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737a3cb-cc0a-4d99-a890-3100fb9c9fb6 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:11.838548 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28797c32-117d-4503-a6fa-f2acf037c00d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:11.906004 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932e100-2b8a-4d44-88eb-53283c77f6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:11.906616 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932e100-2b8a-4d44-88eb-53283c77f6a4 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:11.906886 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932e100-2b8a-4d44-88eb-53283c77f6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:11.907177 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932e100-2b8a-4d44-88eb-53283c77f6a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:11.907889 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c932e100-2b8a-4d44-88eb-53283c77f6a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:02:11.908012 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737a3cb-cc0a-4d99-a890-3100fb9c9fb6 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:11.908194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c932e100-2b8a-4d44-88eb-53283c77f6a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:11.909786 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c932e100-2b8a-4d44-88eb-53283c77f6a4 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:11.912583 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5c2a6-655d-44ce-9ffd-357fcc6a13a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:11.913266 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5c2a6-655d-44ce-9ffd-357fcc6a13a2 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:11.913530 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5c2a6-655d-44ce-9ffd-357fcc6a13a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:11.913802 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5c2a6-655d-44ce-9ffd-357fcc6a13a2 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:11.914526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5c2a6-655d-44ce-9ffd-357fcc6a13a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:11.916442 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5c2a6-655d-44ce-9ffd-357fcc6a13a2 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:11.920853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 779/1551] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:02:11.922039 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16698fc3-c644-4123-ae81-1ecb3532aa12 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:11.922362 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16698fc3-c644-4123-ae81-1ecb3532aa12 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:11.922636 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16698fc3-c644-4123-ae81-1ecb3532aa12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:11.939367 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 773/1552] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:11 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:11.940660 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38194a73-0062-4d8c-8abe-0f6a52d62388 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:11.940950 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38194a73-0062-4d8c-8abe-0f6a52d62388 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:11.941229 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38194a73-0062-4d8c-8abe-0f6a52d62388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:11.943812 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-16698fc3-c644-4123-ae81-1ecb3532aa12 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:11.944994 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-16698fc3-c644-4123-ae81-1ecb3532aa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:11.952914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 780/1553] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:11.953486 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16698fc3-c644-4123-ae81-1ecb3532aa12 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:11.959048 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-38194a73-0062-4d8c-8abe-0f6a52d62388 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:11.960247 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-38194a73-0062-4d8c-8abe-0f6a52d62388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:11.963760 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16698fc3-c644-4123-ae81-1ecb3532aa12 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:11.969210 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 774/1554] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:11.969444 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d561c63-6baa-44ff-ba39-da0c9b92dc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:11.970010 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d561c63-6baa-44ff-ba39-da0c9b92dc7d admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:11.970533 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d561c63-6baa-44ff-ba39-da0c9b92dc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:11.970665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba2efa-10e3-4f65-a65e-c7cf758bc6a4 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:11.970760 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d561c63-6baa-44ff-ba39-da0c9b92dc7d admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:11.970849 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba2efa-10e3-4f65-a65e-c7cf758bc6a4 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:11.971101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba2efa-10e3-4f65-a65e-c7cf758bc6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:11.971356 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d561c63-6baa-44ff-ba39-da0c9b92dc7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:11.972367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: WARNING py.warnings [None req-4d561c63-6baa-44ff-ba39-da0c9b92dc7d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 23:02:11.972367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: warnings.warn(msg) May 09 23:02:11.972367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: May 09 23:02:11.973562 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d561c63-6baa-44ff-ba39-da0c9b92dc7d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:11.997656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 781/1555] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:11 2022] POST /identity/v3/users => generated 290 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:11.997656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6d312-6391-463e-9d5a-6bae3d0cb3b7 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:11.997656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6d312-6391-463e-9d5a-6bae3d0cb3b7 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:11.997656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6d312-6391-463e-9d5a-6bae3d0cb3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:11.999203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ccba2efa-10e3-4f65-a65e-c7cf758bc6a4 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:12.000447 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ccba2efa-10e3-4f65-a65e-c7cf758bc6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:12.009096 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 775/1556] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:12.010178 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa13a31-d945-4c5e-8237-ebce8302cfd0 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.010575 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa13a31-d945-4c5e-8237-ebce8302cfd0 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.010961 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa13a31-d945-4c5e-8237-ebce8302cfd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.017998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-70c6d312-6391-463e-9d5a-6bae3d0cb3b7 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:12.019267 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-70c6d312-6391-463e-9d5a-6bae3d0cb3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:12.026893 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 782/1557] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:12.027637 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6d312-6391-463e-9d5a-6bae3d0cb3b7 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.028077 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-eaa13a31-d945-4c5e-8237-ebce8302cfd0 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:12.029163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-eaa13a31-d945-4c5e-8237-ebce8302cfd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:12.036938 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6d312-6391-463e-9d5a-6bae3d0cb3b7 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.037327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 776/1558] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:12.038279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa13a31-d945-4c5e-8237-ebce8302cfd0 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.042278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c00524-1e1c-4b56-a04e-1451848dd54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.042920 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c00524-1e1c-4b56-a04e-1451848dd54f admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.043196 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c00524-1e1c-4b56-a04e-1451848dd54f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.043464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c00524-1e1c-4b56-a04e-1451848dd54f admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.044144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c00524-1e1c-4b56-a04e-1451848dd54f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.046097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c00524-1e1c-4b56-a04e-1451848dd54f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.049306 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa13a31-d945-4c5e-8237-ebce8302cfd0 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.056921 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9107a02-5d56-4319-8c15-75b77f58a98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.057582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9107a02-5d56-4319-8c15-75b77f58a98f admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.057864 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9107a02-5d56-4319-8c15-75b77f58a98f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.058169 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9107a02-5d56-4319-8c15-75b77f58a98f admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.058946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9107a02-5d56-4319-8c15-75b77f58a98f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.060564 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9107a02-5d56-4319-8c15-75b77f58a98f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.071800 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 783/1559] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:12 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:12.072354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 777/1560] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:12.073773 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59882d52-7704-4d5b-9775-679d90520cd5 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.074067 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59882d52-7704-4d5b-9775-679d90520cd5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.074368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59882d52-7704-4d5b-9775-679d90520cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.075886 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8afa6-5107-4937-9037-0fefc2fcaca7 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.076280 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8afa6-5107-4937-9037-0fefc2fcaca7 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.076557 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8afa6-5107-4937-9037-0fefc2fcaca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.093279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-59882d52-7704-4d5b-9775-679d90520cd5 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:12.093488 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-95b8afa6-5107-4937-9037-0fefc2fcaca7 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:12.094563 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-59882d52-7704-4d5b-9775-679d90520cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:12.094701 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-95b8afa6-5107-4937-9037-0fefc2fcaca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:12.157649 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 778/1561] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:12.158437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59882d52-7704-4d5b-9775-679d90520cd5 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.159889 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 784/1562] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:12.161230 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8afa6-5107-4937-9037-0fefc2fcaca7 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.168062 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59882d52-7704-4d5b-9775-679d90520cd5 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.170336 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8afa6-5107-4937-9037-0fefc2fcaca7 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.172639 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca262ae3-c649-4674-8642-add506b4d078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.173872 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca262ae3-c649-4674-8642-add506b4d078 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.174105 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca262ae3-c649-4674-8642-add506b4d078 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.174417 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca262ae3-c649-4674-8642-add506b4d078 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.175131 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca262ae3-c649-4674-8642-add506b4d078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.175408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15394119-e144-4956-bb7d-8a5f94aaa7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.176047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15394119-e144-4956-bb7d-8a5f94aaa7ea admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.176358 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15394119-e144-4956-bb7d-8a5f94aaa7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.176635 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15394119-e144-4956-bb7d-8a5f94aaa7ea admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.176746 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca262ae3-c649-4674-8642-add506b4d078 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.177351 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15394119-e144-4956-bb7d-8a5f94aaa7ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:02:12.177655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15394119-e144-4956-bb7d-8a5f94aaa7ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.179264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15394119-e144-4956-bb7d-8a5f94aaa7ea admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.186725 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 779/1563] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:12.188026 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5e47b-a152-4253-ab6f-37bf3d3591ee None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.188382 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5e47b-a152-4253-ab6f-37bf3d3591ee None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.189293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5e47b-a152-4253-ab6f-37bf3d3591ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.192060 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 785/1564] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:02:12.193326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b465a-b17a-4881-bb21-5bc1d4871bf9 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.193624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b465a-b17a-4881-bb21-5bc1d4871bf9 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.193929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b465a-b17a-4881-bb21-5bc1d4871bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.210469 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-18d5e47b-a152-4253-ab6f-37bf3d3591ee None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:12.211785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-18d5e47b-a152-4253-ab6f-37bf3d3591ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:12.212222 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-892b465a-b17a-4881-bb21-5bc1d4871bf9 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:12.213387 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-892b465a-b17a-4881-bb21-5bc1d4871bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:12.219699 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 780/1565] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:12.220656 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5e47b-a152-4253-ab6f-37bf3d3591ee None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.224604 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 786/1566] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:12.225494 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b465a-b17a-4881-bb21-5bc1d4871bf9 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.232548 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5e47b-a152-4253-ab6f-37bf3d3591ee None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.235112 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b465a-b17a-4881-bb21-5bc1d4871bf9 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.237559 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe66324-c9c2-4005-9b47-99cfe5ce5e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.238298 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe66324-c9c2-4005-9b47-99cfe5ce5e03 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.238575 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe66324-c9c2-4005-9b47-99cfe5ce5e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.238848 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe66324-c9c2-4005-9b47-99cfe5ce5e03 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.239515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe66324-c9c2-4005-9b47-99cfe5ce5e03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.240279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55835028-554f-44a0-93be-aa1f9e8812f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.240979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55835028-554f-44a0-93be-aa1f9e8812f2 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.241259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55835028-554f-44a0-93be-aa1f9e8812f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.241395 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe66324-c9c2-4005-9b47-99cfe5ce5e03 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.241533 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55835028-554f-44a0-93be-aa1f9e8812f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.242287 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55835028-554f-44a0-93be-aa1f9e8812f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.243916 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55835028-554f-44a0-93be-aa1f9e8812f2 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.254305 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 787/1567] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:12.255117 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55835028-554f-44a0-93be-aa1f9e8812f2 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.267738 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 781/1568] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:12 2022] POST /identity/v3/users => generated 290 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:12.267738 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe66324-c9c2-4005-9b47-99cfe5ce5e03 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.268022 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55835028-554f-44a0-93be-aa1f9e8812f2 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.271138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60a54b-b5df-40ca-ad00-ef634ea701b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.271772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60a54b-b5df-40ca-ad00-ef634ea701b4 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.272057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60a54b-b5df-40ca-ad00-ef634ea701b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.272319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60a54b-b5df-40ca-ad00-ef634ea701b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.273031 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60a54b-b5df-40ca-ad00-ef634ea701b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:02:12.273334 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60a54b-b5df-40ca-ad00-ef634ea701b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.274945 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60a54b-b5df-40ca-ad00-ef634ea701b4 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.276705 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe66324-c9c2-4005-9b47-99cfe5ce5e03 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.281127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ee22f-80a5-426c-86fc-780c25048349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.281752 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ee22f-80a5-426c-86fc-780c25048349 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.282029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ee22f-80a5-426c-86fc-780c25048349 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.282834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ee22f-80a5-426c-86fc-780c25048349 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.283610 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ee22f-80a5-426c-86fc-780c25048349 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:02:12.283909 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ee22f-80a5-426c-86fc-780c25048349 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.285603 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ee22f-80a5-426c-86fc-780c25048349 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.287692 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 788/1569] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:02:12.288499 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60a54b-b5df-40ca-ad00-ef634ea701b4 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.297629 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 782/1570] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:02:12.299036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ee22f-80a5-426c-86fc-780c25048349 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.304381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60a54b-b5df-40ca-ad00-ef634ea701b4 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.308275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ee22f-80a5-426c-86fc-780c25048349 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.309276 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bef837-7a95-4325-9bf3-a40dd25c886a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.309948 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bef837-7a95-4325-9bf3-a40dd25c886a admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.310266 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bef837-7a95-4325-9bf3-a40dd25c886a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.310546 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bef837-7a95-4325-9bf3-a40dd25c886a admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.311262 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bef837-7a95-4325-9bf3-a40dd25c886a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.312788 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c19dc3-d271-4534-b617-e9ec202eb3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.312914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bef837-7a95-4325-9bf3-a40dd25c886a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.313436 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c19dc3-d271-4534-b617-e9ec202eb3e0 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.313721 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c19dc3-d271-4534-b617-e9ec202eb3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.314093 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c19dc3-d271-4534-b617-e9ec202eb3e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.314713 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c19dc3-d271-4534-b617-e9ec202eb3e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.316579 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c19dc3-d271-4534-b617-e9ec202eb3e0 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.322780 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 789/1571] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:12.323570 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bef837-7a95-4325-9bf3-a40dd25c886a admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.333386 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bef837-7a95-4325-9bf3-a40dd25c886a admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.338472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5248d6-bb89-45e8-9826-b138c5d94075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.339108 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5248d6-bb89-45e8-9826-b138c5d94075 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.339390 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5248d6-bb89-45e8-9826-b138c5d94075 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.339668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5248d6-bb89-45e8-9826-b138c5d94075 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.340368 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5248d6-bb89-45e8-9826-b138c5d94075 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:02:12.340674 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5248d6-bb89-45e8-9826-b138c5d94075 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.342308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 783/1572] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:12 2022] POST /identity/v3/projects => generated 391 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:12.342448 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5248d6-bb89-45e8-9826-b138c5d94075 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.343003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c19dc3-d271-4534-b617-e9ec202eb3e0 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.363996 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 790/1573] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:02:12.363996 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5248d6-bb89-45e8-9826-b138c5d94075 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.373094 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5248d6-bb89-45e8-9826-b138c5d94075 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.400307 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c19dc3-d271-4534-b617-e9ec202eb3e0 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.404878 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074405a-69bf-4f75-bf45-63a1b506558a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.405525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074405a-69bf-4f75-bf45-63a1b506558a admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.405804 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074405a-69bf-4f75-bf45-63a1b506558a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.406071 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074405a-69bf-4f75-bf45-63a1b506558a admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.406811 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074405a-69bf-4f75-bf45-63a1b506558a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:02:12.407108 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074405a-69bf-4f75-bf45-63a1b506558a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.409271 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074405a-69bf-4f75-bf45-63a1b506558a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.420010 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 784/1574] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:02:12.420010 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074405a-69bf-4f75-bf45-63a1b506558a admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.429707 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a8507-d1c7-4db8-b48a-629fb54bfb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.430662 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074405a-69bf-4f75-bf45-63a1b506558a admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.430790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a8507-d1c7-4db8-b48a-629fb54bfb15 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.430999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a8507-d1c7-4db8-b48a-629fb54bfb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.431283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a8507-d1c7-4db8-b48a-629fb54bfb15 admin admin] PATH_INFO: `/v3/projects/b776734a879d4c0785b1d3728b79b766/users/84a45506c19c4cda8173c164a097f925/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.435342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c0379-9f6f-48df-a57c-569b44808dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.435969 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c0379-9f6f-48df-a57c-569b44808dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.436244 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c0379-9f6f-48df-a57c-569b44808dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.436511 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c0379-9f6f-48df-a57c-569b44808dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.437324 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c0379-9f6f-48df-a57c-569b44808dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.438940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c0379-9f6f-48df-a57c-569b44808dc1 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.445322 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a8507-d1c7-4db8-b48a-629fb54bfb15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b776734a879d4c0785b1d3728b79b766, user_id=84a45506c19c4cda8173c164a097f925, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.448067 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a8507-d1c7-4db8-b48a-629fb54bfb15 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.448835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 785/1575] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:12.449573 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c0379-9f6f-48df-a57c-569b44808dc1 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.458840 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c0379-9f6f-48df-a57c-569b44808dc1 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.460850 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 791/1576] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:12 2022] PUT /identity/v3/projects/b776734a879d4c0785b1d3728b79b766/users/84a45506c19c4cda8173c164a097f925/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:12.461704 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a8507-d1c7-4db8-b48a-629fb54bfb15 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.514453 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc6612-7395-4f22-9245-8debf5b8266e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.515112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc6612-7395-4f22-9245-8debf5b8266e admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.515392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc6612-7395-4f22-9245-8debf5b8266e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.515663 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc6612-7395-4f22-9245-8debf5b8266e admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.516344 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc6612-7395-4f22-9245-8debf5b8266e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.518280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc6612-7395-4f22-9245-8debf5b8266e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.521063 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a8507-d1c7-4db8-b48a-629fb54bfb15 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.525640 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c43dc-234d-4965-be91-e6b68dddf3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.526305 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c43dc-234d-4965-be91-e6b68dddf3d9 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.526578 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c43dc-234d-4965-be91-e6b68dddf3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.526844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c43dc-234d-4965-be91-e6b68dddf3d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.527512 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c43dc-234d-4965-be91-e6b68dddf3d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.529377 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c43dc-234d-4965-be91-e6b68dddf3d9 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.543136 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 786/1577] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:12 2022] POST /identity/v3/projects => generated 405 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:12.544173 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc6612-7395-4f22-9245-8debf5b8266e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.558732 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 792/1578] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:12 2022] POST /identity/v3/projects => generated 377 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:12.558732 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c43dc-234d-4965-be91-e6b68dddf3d9 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.609029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc6612-7395-4f22-9245-8debf5b8266e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.614157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca60ba-efea-4513-8ff5-eaf2dc38c502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.615121 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca60ba-efea-4513-8ff5-eaf2dc38c502 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.615406 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca60ba-efea-4513-8ff5-eaf2dc38c502 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.615679 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca60ba-efea-4513-8ff5-eaf2dc38c502 admin admin] PATH_INFO: `/v3/projects/a89b506fb2784f79bb7fac2065cc6761/users/6be2c71368ee41f383832bb693969e89/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.616242 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c43dc-234d-4965-be91-e6b68dddf3d9 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.620740 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc59a97-757b-44cb-9276-cb70af6c1808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.621397 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc59a97-757b-44cb-9276-cb70af6c1808 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.621677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc59a97-757b-44cb-9276-cb70af6c1808 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.621952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc59a97-757b-44cb-9276-cb70af6c1808 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.622665 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc59a97-757b-44cb-9276-cb70af6c1808 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.624574 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc59a97-757b-44cb-9276-cb70af6c1808 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.629846 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca60ba-efea-4513-8ff5-eaf2dc38c502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a89b506fb2784f79bb7fac2065cc6761, user_id=6be2c71368ee41f383832bb693969e89, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.630974 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: WARNING py.warnings [None req-d0ca60ba-efea-4513-8ff5-eaf2dc38c502 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 23:02:12.630974 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: warnings.warn(msg) May 09 23:02:12.630974 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: May 09 23:02:12.632953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca60ba-efea-4513-8ff5-eaf2dc38c502 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.648777 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 787/1579] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:12 2022] PUT /identity/v3/projects/a89b506fb2784f79bb7fac2065cc6761/users/6be2c71368ee41f383832bb693969e89/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:12.649733 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca60ba-efea-4513-8ff5-eaf2dc38c502 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.655287 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 793/1580] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:12 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:12.656468 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc59a97-757b-44cb-9276-cb70af6c1808 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.667859 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc59a97-757b-44cb-9276-cb70af6c1808 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.709799 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca60ba-efea-4513-8ff5-eaf2dc38c502 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.714612 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e9827-4725-46fe-935e-74def690894b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.715377 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e9827-4725-46fe-935e-74def690894b admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.715764 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e9827-4725-46fe-935e-74def690894b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.716117 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e9827-4725-46fe-935e-74def690894b admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.716805 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e9827-4725-46fe-935e-74def690894b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.718755 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e9827-4725-46fe-935e-74def690894b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.721574 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1fc51-47b3-42e9-aa9e-5fd54f5f182b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.722280 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1fc51-47b3-42e9-aa9e-5fd54f5f182b admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.722566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1fc51-47b3-42e9-aa9e-5fd54f5f182b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.722844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1fc51-47b3-42e9-aa9e-5fd54f5f182b admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.723522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1fc51-47b3-42e9-aa9e-5fd54f5f182b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.725447 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1fc51-47b3-42e9-aa9e-5fd54f5f182b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.745781 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 788/1581] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:12 2022] POST /identity/v3/projects => generated 405 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:12.746656 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e9827-4725-46fe-935e-74def690894b admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.751710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 794/1582] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:12 2022] POST /identity/v3/projects => generated 369 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:12.753104 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b977e7-9de8-4417-aa21-c71aca0906f6 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.753395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b977e7-9de8-4417-aa21-c71aca0906f6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.753669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b977e7-9de8-4417-aa21-c71aca0906f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.770454 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e0b977e7-9de8-4417-aa21-c71aca0906f6 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:12.771663 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e0b977e7-9de8-4417-aa21-c71aca0906f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:12.808089 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e9827-4725-46fe-935e-74def690894b admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.830942 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 795/1583] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:12.831759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b977e7-9de8-4417-aa21-c71aca0906f6 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.841254 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b977e7-9de8-4417-aa21-c71aca0906f6 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.845796 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc44190-9764-46b3-8ac7-64aaee08b484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.846477 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc44190-9764-46b3-8ac7-64aaee08b484 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.846759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc44190-9764-46b3-8ac7-64aaee08b484 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.847029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc44190-9764-46b3-8ac7-64aaee08b484 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.847704 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc44190-9764-46b3-8ac7-64aaee08b484 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.849633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc44190-9764-46b3-8ac7-64aaee08b484 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.866709 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79759428-ef8a-411b-b021-9fcbaa4d56d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.867379 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79759428-ef8a-411b-b021-9fcbaa4d56d6 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.867661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79759428-ef8a-411b-b021-9fcbaa4d56d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.867933 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79759428-ef8a-411b-b021-9fcbaa4d56d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.868647 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79759428-ef8a-411b-b021-9fcbaa4d56d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.870269 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79759428-ef8a-411b-b021-9fcbaa4d56d6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.870930 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 796/1584] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:12 2022] POST /identity/v3/users => generated 303 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:12.871624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc44190-9764-46b3-8ac7-64aaee08b484 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.881234 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 789/1585] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:12.881388 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc44190-9764-46b3-8ac7-64aaee08b484 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.882036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79759428-ef8a-411b-b021-9fcbaa4d56d6 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.887981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a7a02-2722-4a6a-9341-4e98de266d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.888640 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a7a02-2722-4a6a-9341-4e98de266d05 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.888922 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a7a02-2722-4a6a-9341-4e98de266d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.889202 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a7a02-2722-4a6a-9341-4e98de266d05 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.889892 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a7a02-2722-4a6a-9341-4e98de266d05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.891887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a7a02-2722-4a6a-9341-4e98de266d05 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.892913 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79759428-ef8a-411b-b021-9fcbaa4d56d6 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.897486 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2eadf5-c869-4585-a565-e642f4f75730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.898108 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2eadf5-c869-4585-a565-e642f4f75730 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.898981 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2eadf5-c869-4585-a565-e642f4f75730 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.899251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2eadf5-c869-4585-a565-e642f4f75730 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.900097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2eadf5-c869-4585-a565-e642f4f75730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.901519 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2eadf5-c869-4585-a565-e642f4f75730 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.912313 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 797/1586] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:12 2022] POST /identity/v3/users => generated 289 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:12.915902 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 790/1587] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:12.916052 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa841fd-6761-46bf-80ba-c1914a8e11c6 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.916052 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa841fd-6761-46bf-80ba-c1914a8e11c6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.916230 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2eadf5-c869-4585-a565-e642f4f75730 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.916320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa841fd-6761-46bf-80ba-c1914a8e11c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.924863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2eadf5-c869-4585-a565-e642f4f75730 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.929309 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5a5ce-6703-4538-8057-df3cd849fbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.929938 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5a5ce-6703-4538-8057-df3cd849fbec admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.930251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5a5ce-6703-4538-8057-df3cd849fbec admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.930524 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5a5ce-6703-4538-8057-df3cd849fbec admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.931188 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5a5ce-6703-4538-8057-df3cd849fbec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.933096 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5a5ce-6703-4538-8057-df3cd849fbec admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.941237 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-faa841fd-6761-46bf-80ba-c1914a8e11c6 None None] MFA Rules not processed for user `6be2c71368ee41f383832bb693969e89`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:12.942680 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-faa841fd-6761-46bf-80ba-c1914a8e11c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:12.952431 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 791/1588] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:12 2022] POST /identity/v3/users => generated 303 bytes in 38 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:12.953278 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5a5ce-6703-4538-8057-df3cd849fbec admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:12.962643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5a5ce-6703-4538-8057-df3cd849fbec admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:12.967155 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b606fb-0b03-4651-8729-4c7a310095d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:12.967771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b606fb-0b03-4651-8729-4c7a310095d6 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:12.968057 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b606fb-0b03-4651-8729-4c7a310095d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:12.968331 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b606fb-0b03-4651-8729-4c7a310095d6 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:12.968998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b606fb-0b03-4651-8729-4c7a310095d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:12.970933 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b606fb-0b03-4651-8729-4c7a310095d6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:12.990541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 792/1589] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:12 2022] POST /identity/v3/users => generated 285 bytes in 38 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:12.991436 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b606fb-0b03-4651-8729-4c7a310095d6 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:13.007022 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 798/1590] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:13.007952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa841fd-6761-46bf-80ba-c1914a8e11c6 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:13.017989 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa841fd-6761-46bf-80ba-c1914a8e11c6 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:13.022576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45d931-3e4d-43d8-ba20-3f54e7743e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:13.023219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45d931-3e4d-43d8-ba20-3f54e7743e63 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.023500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45d931-3e4d-43d8-ba20-3f54e7743e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.023804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45d931-3e4d-43d8-ba20-3f54e7743e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:13.024519 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45d931-3e4d-43d8-ba20-3f54e7743e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:13.026103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45d931-3e4d-43d8-ba20-3f54e7743e63 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:13.036119 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 799/1591] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:13.036880 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45d931-3e4d-43d8-ba20-3f54e7743e63 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:13.044124 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b606fb-0b03-4651-8729-4c7a310095d6 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:13.046091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45d931-3e4d-43d8-ba20-3f54e7743e63 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:13.047864 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df97ce-8956-48a8-a292-d1831d71295d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:13.049079 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df97ce-8956-48a8-a292-d1831d71295d service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.049304 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df97ce-8956-48a8-a292-d1831d71295d service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.049578 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df97ce-8956-48a8-a292-d1831d71295d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:13.051478 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcadf1-c2cb-4539-bbcd-d906f9e3998c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:13.052311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcadf1-c2cb-4539-bbcd-d906f9e3998c admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.052589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcadf1-c2cb-4539-bbcd-d906f9e3998c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.052866 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcadf1-c2cb-4539-bbcd-d906f9e3998c admin admin] PATH_INFO: `/v3/projects/192f40de826942b1994a0e8ea5db9d88/users/58586c1f929f4218abdf6862a092f83d/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:13.058317 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04df97ce-8956-48a8-a292-d1831d71295d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:13.059741 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04df97ce-8956-48a8-a292-d1831d71295d service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:13.067560 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcadf1-c2cb-4539-bbcd-d906f9e3998c admin admin] RBAC: Authorizing `identity:create_grant(project_id=192f40de826942b1994a0e8ea5db9d88, user_id=58586c1f929f4218abdf6862a092f83d, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:13.070250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcadf1-c2cb-4539-bbcd-d906f9e3998c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:13.071860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 793/1592] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:13.072855 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df97ce-8956-48a8-a292-d1831d71295d service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:13.082955 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df97ce-8956-48a8-a292-d1831d71295d service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:13.085602 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 800/1593] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:13 2022] PUT /identity/v3/projects/192f40de826942b1994a0e8ea5db9d88/users/58586c1f929f4218abdf6862a092f83d/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:13.086721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcadf1-c2cb-4539-bbcd-d906f9e3998c admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:13.143135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8342d-27f7-4f31-9e81-31ffe112e17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:13.143823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8342d-27f7-4f31-9e81-31ffe112e17f admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.144105 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8342d-27f7-4f31-9e81-31ffe112e17f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.144382 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8342d-27f7-4f31-9e81-31ffe112e17f admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:13.145111 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8342d-27f7-4f31-9e81-31ffe112e17f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:13.146871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8342d-27f7-4f31-9e81-31ffe112e17f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:13.157514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 794/1594] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:13.158397 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8342d-27f7-4f31-9e81-31ffe112e17f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:13.159903 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcadf1-c2cb-4539-bbcd-d906f9e3998c admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:13.164933 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ed094-17df-4aae-92ef-fd22b0681ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:13.165765 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ed094-17df-4aae-92ef-fd22b0681ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.166175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ed094-17df-4aae-92ef-fd22b0681ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.166588 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ed094-17df-4aae-92ef-fd22b0681ca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:13.167406 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ed094-17df-4aae-92ef-fd22b0681ca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:13.168552 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8342d-27f7-4f31-9e81-31ffe112e17f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:13.169232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ed094-17df-4aae-92ef-fd22b0681ca5 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:13.173379 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b550cfc-bd6f-4163-a71f-9adcca08ebc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:13.174058 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b550cfc-bd6f-4163-a71f-9adcca08ebc6 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.174375 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b550cfc-bd6f-4163-a71f-9adcca08ebc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.174651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b550cfc-bd6f-4163-a71f-9adcca08ebc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:13.175435 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b550cfc-bd6f-4163-a71f-9adcca08ebc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:13.177001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b550cfc-bd6f-4163-a71f-9adcca08ebc6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:13.184632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 801/1595] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:13.184632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ed094-17df-4aae-92ef-fd22b0681ca5 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:13.192794 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 795/1596] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:13.193636 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b550cfc-bd6f-4163-a71f-9adcca08ebc6 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:13.197603 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ed094-17df-4aae-92ef-fd22b0681ca5 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:13.203852 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5071e6-3fd5-42cf-9e54-93da4cc96f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:13.205486 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5071e6-3fd5-42cf-9e54-93da4cc96f64 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.205935 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b550cfc-bd6f-4163-a71f-9adcca08ebc6 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:13.206185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5071e6-3fd5-42cf-9e54-93da4cc96f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.206713 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5071e6-3fd5-42cf-9e54-93da4cc96f64 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:13.207596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5071e6-3fd5-42cf-9e54-93da4cc96f64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:13.209578 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5071e6-3fd5-42cf-9e54-93da4cc96f64 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:13.212529 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd26148-459e-4ac0-a18b-e586b99ed419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:13.213386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd26148-459e-4ac0-a18b-e586b99ed419 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.214252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd26148-459e-4ac0-a18b-e586b99ed419 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.214550 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd26148-459e-4ac0-a18b-e586b99ed419 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:13.215252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd26148-459e-4ac0-a18b-e586b99ed419 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:13.217330 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd26148-459e-4ac0-a18b-e586b99ed419 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:13.223510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 802/1597] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:13.224642 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5071e6-3fd5-42cf-9e54-93da4cc96f64 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:13.237361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5071e6-3fd5-42cf-9e54-93da4cc96f64 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:13.243763 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b57002b-de68-4c17-8a94-b6508c30caa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:13.244667 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57002b-de68-4c17-8a94-b6508c30caa7 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.245207 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57002b-de68-4c17-8a94-b6508c30caa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.245640 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57002b-de68-4c17-8a94-b6508c30caa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:13.246570 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b57002b-de68-4c17-8a94-b6508c30caa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:13.248402 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b57002b-de68-4c17-8a94-b6508c30caa7 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:13.260206 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 796/1598] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:13 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:13.265017 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 803/1599] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:13.265193 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97feb64-c5df-40a4-b88f-017acecffb38 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.265193 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97feb64-c5df-40a4-b88f-017acecffb38 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.265403 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b57002b-de68-4c17-8a94-b6508c30caa7 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:13.265514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97feb64-c5df-40a4-b88f-017acecffb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:13.275692 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b57002b-de68-4c17-8a94-b6508c30caa7 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:13.297886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-f97feb64-c5df-40a4-b88f-017acecffb38 None None] MFA Rules not processed for user `58586c1f929f4218abdf6862a092f83d`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:13.299806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f97feb64-c5df-40a4-b88f-017acecffb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:13.349550 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12fbd4-9712-4951-8203-bb1cd1ee15cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:13.350300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12fbd4-9712-4951-8203-bb1cd1ee15cd admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.350582 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12fbd4-9712-4951-8203-bb1cd1ee15cd admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.350855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12fbd4-9712-4951-8203-bb1cd1ee15cd admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:13.351569 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12fbd4-9712-4951-8203-bb1cd1ee15cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:13.353165 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12fbd4-9712-4951-8203-bb1cd1ee15cd admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:13.364131 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 804/1600] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:13.365054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12fbd4-9712-4951-8203-bb1cd1ee15cd admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:13.375034 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12fbd4-9712-4951-8203-bb1cd1ee15cd admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:13.380841 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed19da-2028-4476-8537-9c56cc229d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:13.381697 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed19da-2028-4476-8537-9c56cc229d68 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.381975 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed19da-2028-4476-8537-9c56cc229d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.382273 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed19da-2028-4476-8537-9c56cc229d68 admin admin] PATH_INFO: `/v3/projects/0cb055ff9f0d4af6962070c7a2f19bf3/users/e41b4feaa9be44e89210b70c19ef10ab/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:13.382804 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 797/1601] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:13.383477 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97feb64-c5df-40a4-b88f-017acecffb38 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:13.395365 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97feb64-c5df-40a4-b88f-017acecffb38 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:13.399502 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed19da-2028-4476-8537-9c56cc229d68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cb055ff9f0d4af6962070c7a2f19bf3, user_id=e41b4feaa9be44e89210b70c19ef10ab, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:13.401314 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70008d5b-26d6-47ef-98fa-b394265ebb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:13.402263 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70008d5b-26d6-47ef-98fa-b394265ebb82 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.402601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70008d5b-26d6-47ef-98fa-b394265ebb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.402810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed19da-2028-4476-8537-9c56cc229d68 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:13.402978 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70008d5b-26d6-47ef-98fa-b394265ebb82 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:13.403703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70008d5b-26d6-47ef-98fa-b394265ebb82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:13.406073 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70008d5b-26d6-47ef-98fa-b394265ebb82 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:13.417384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 805/1602] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:13 2022] PUT /identity/v3/projects/0cb055ff9f0d4af6962070c7a2f19bf3/users/e41b4feaa9be44e89210b70c19ef10ab/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:13.418097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed19da-2028-4476-8537-9c56cc229d68 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:13.422246 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 798/1603] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:13.424153 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70008d5b-26d6-47ef-98fa-b394265ebb82 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:13.433738 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70008d5b-26d6-47ef-98fa-b394265ebb82 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:13.492011 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed19da-2028-4476-8537-9c56cc229d68 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:13.499285 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f80b87-3b15-44e4-ad5a-f643a654c85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:13.499967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f80b87-3b15-44e4-ad5a-f643a654c85f admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.500246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f80b87-3b15-44e4-ad5a-f643a654c85f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.500572 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f80b87-3b15-44e4-ad5a-f643a654c85f admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:13.502059 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f80b87-3b15-44e4-ad5a-f643a654c85f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:13.503550 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58f7a0-7265-48dc-82e2-29ee115b2548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:13.503939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f80b87-3b15-44e4-ad5a-f643a654c85f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:13.504437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58f7a0-7265-48dc-82e2-29ee115b2548 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.504958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58f7a0-7265-48dc-82e2-29ee115b2548 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.505325 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58f7a0-7265-48dc-82e2-29ee115b2548 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:13.506196 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a58f7a0-7265-48dc-82e2-29ee115b2548 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:13.508243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a58f7a0-7265-48dc-82e2-29ee115b2548 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:13.516670 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 806/1604] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:13.518809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f80b87-3b15-44e4-ad5a-f643a654c85f admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:13.531642 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f80b87-3b15-44e4-ad5a-f643a654c85f admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:13.533698 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 799/1605] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:13 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:13.534715 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58f7a0-7265-48dc-82e2-29ee115b2548 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:13.541759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbe8c7-6340-466a-ac9e-2ac3528cd282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:13.542765 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbe8c7-6340-466a-ac9e-2ac3528cd282 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.543336 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbe8c7-6340-466a-ac9e-2ac3528cd282 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.543756 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbe8c7-6340-466a-ac9e-2ac3528cd282 admin admin] PATH_INFO: `/v3/projects/02f99bf5f6404a1fa6c45c21136c3889/users/3dc4b31fcb154251ac27e4e4dce5e6c7/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:13.545578 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58f7a0-7265-48dc-82e2-29ee115b2548 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:13.552149 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99dbd3-6aba-4954-bc93-be9739741ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:13.553341 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99dbd3-6aba-4954-bc93-be9739741ee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.554241 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99dbd3-6aba-4954-bc93-be9739741ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.554241 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99dbd3-6aba-4954-bc93-be9739741ee8 admin admin] PATH_INFO: `/v3/projects/0805f702593a4c379db7cdeec818f94b/users/90018602e07f4a4ca5f46c5c85bc818e/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:13.571866 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbe8c7-6340-466a-ac9e-2ac3528cd282 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02f99bf5f6404a1fa6c45c21136c3889, user_id=3dc4b31fcb154251ac27e4e4dce5e6c7, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:13.572402 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99dbd3-6aba-4954-bc93-be9739741ee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0805f702593a4c379db7cdeec818f94b, user_id=90018602e07f4a4ca5f46c5c85bc818e, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:13.574904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbe8c7-6340-466a-ac9e-2ac3528cd282 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:13.575549 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99dbd3-6aba-4954-bc93-be9739741ee8 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:13.591877 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 807/1606] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:13 2022] PUT /identity/v3/projects/02f99bf5f6404a1fa6c45c21136c3889/users/3dc4b31fcb154251ac27e4e4dce5e6c7/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:13.592630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 800/1607] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:13 2022] PUT /identity/v3/projects/0805f702593a4c379db7cdeec818f94b/users/90018602e07f4a4ca5f46c5c85bc818e/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:13.592750 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbe8c7-6340-466a-ac9e-2ac3528cd282 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:13.595482 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19bed9-8f1a-4d1b-accb-cc003520f07f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.595762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19bed9-8f1a-4d1b-accb-cc003520f07f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.596240 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19bed9-8f1a-4d1b-accb-cc003520f07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:13.615999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-8e19bed9-8f1a-4d1b-accb-cc003520f07f None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:13.617246 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8e19bed9-8f1a-4d1b-accb-cc003520f07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:13.675926 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbe8c7-6340-466a-ac9e-2ac3528cd282 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:13.704153 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 801/1608] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:13.705399 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19bed9-8f1a-4d1b-accb-cc003520f07f None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:13.718089 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19bed9-8f1a-4d1b-accb-cc003520f07f None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:13.723084 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006bfba-a212-4c16-878f-5d195a4fad93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:13.723969 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006bfba-a212-4c16-878f-5d195a4fad93 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.724639 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006bfba-a212-4c16-878f-5d195a4fad93 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.724918 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006bfba-a212-4c16-878f-5d195a4fad93 admin admin] PATH_INFO: `/v3/projects/b398b8a010f548a28e72c7dd20789571/users/461bf24c5b1a429f8f52b036fe768b5d/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:13.741289 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a006bfba-a212-4c16-878f-5d195a4fad93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b398b8a010f548a28e72c7dd20789571, user_id=461bf24c5b1a429f8f52b036fe768b5d, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:13.744011 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a006bfba-a212-4c16-878f-5d195a4fad93 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:13.744923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625b2f3-c86a-4cee-a846-5674ffe4f79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:13.745662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625b2f3-c86a-4cee-a846-5674ffe4f79a admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.746014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625b2f3-c86a-4cee-a846-5674ffe4f79a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.746396 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625b2f3-c86a-4cee-a846-5674ffe4f79a admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:13.747092 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625b2f3-c86a-4cee-a846-5674ffe4f79a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:13.749063 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625b2f3-c86a-4cee-a846-5674ffe4f79a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:13.759431 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 802/1609] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:13 2022] PUT /identity/v3/projects/b398b8a010f548a28e72c7dd20789571/users/461bf24c5b1a429f8f52b036fe768b5d/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:13.763539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006bfba-a212-4c16-878f-5d195a4fad93 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:13.776966 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 808/1610] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:13 2022] POST /identity/v3/projects => generated 381 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:13.777797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625b2f3-c86a-4cee-a846-5674ffe4f79a admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:13.834024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006bfba-a212-4c16-878f-5d195a4fad93 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:13.855174 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625b2f3-c86a-4cee-a846-5674ffe4f79a admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:13.861206 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f3ac2-4f95-4a97-abba-0e70e5e7ce25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:13.862020 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f3ac2-4f95-4a97-abba-0e70e5e7ce25 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.862169 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f3ac2-4f95-4a97-abba-0e70e5e7ce25 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.862452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f3ac2-4f95-4a97-abba-0e70e5e7ce25 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:13.863326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f3ac2-4f95-4a97-abba-0e70e5e7ce25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:13.865034 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f3ac2-4f95-4a97-abba-0e70e5e7ce25 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:13.876018 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 809/1611] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:13.877324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e3eb4-6373-4e84-b6e6-6c077c0e740d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.877611 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e3eb4-6373-4e84-b6e6-6c077c0e740d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.877888 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e3eb4-6373-4e84-b6e6-6c077c0e740d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:13.897903 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-834e3eb4-6373-4e84-b6e6-6c077c0e740d None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:13.899226 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-834e3eb4-6373-4e84-b6e6-6c077c0e740d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:13.911662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 810/1612] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:13.912837 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d56377-2b5e-462a-a398-badec68d0e62 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.913113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d56377-2b5e-462a-a398-badec68d0e62 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.913446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d56377-2b5e-462a-a398-badec68d0e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:13.932520 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae39a2-7e16-499f-a9a9-ca5d96d060dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:13.934359 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae39a2-7e16-499f-a9a9-ca5d96d060dd admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.934683 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae39a2-7e16-499f-a9a9-ca5d96d060dd admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.934963 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae39a2-7e16-499f-a9a9-ca5d96d060dd admin admin] PATH_INFO: `/v3/projects/ee43ae34b4cd499c98fa333ee26bb26d/users/10b083e6ebaf458da7a6c52c80a4423d/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:13.945976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-00d56377-2b5e-462a-a398-badec68d0e62 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:13.947225 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-00d56377-2b5e-462a-a398-badec68d0e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:13.955220 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae39a2-7e16-499f-a9a9-ca5d96d060dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee43ae34b4cd499c98fa333ee26bb26d, user_id=10b083e6ebaf458da7a6c52c80a4423d, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:13.957999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae39a2-7e16-499f-a9a9-ca5d96d060dd admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:13.961522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 811/1613] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:13.962735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d56377-2b5e-462a-a398-badec68d0e62 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:13.970838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-00d56377-2b5e-462a-a398-badec68d0e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:13.979940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 803/1614] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:13 2022] PUT /identity/v3/projects/ee43ae34b4cd499c98fa333ee26bb26d/users/10b083e6ebaf458da7a6c52c80a4423d/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:13.981510 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce5eef-f251-47ae-b80c-8fd10e4b88dc None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:13.981982 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce5eef-f251-47ae-b80c-8fd10e4b88dc None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:13.982684 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce5eef-f251-47ae-b80c-8fd10e4b88dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:14.017369 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-0bce5eef-f251-47ae-b80c-8fd10e4b88dc None None] MFA Rules not processed for user `461bf24c5b1a429f8f52b036fe768b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:14.019058 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0bce5eef-f251-47ae-b80c-8fd10e4b88dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:14.048879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d56377-2b5e-462a-a398-badec68d0e62 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:14.113612 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbabb29e-2f9a-454c-b149-468ca462a382 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:14.114466 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbabb29e-2f9a-454c-b149-468ca462a382 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:14.114751 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbabb29e-2f9a-454c-b149-468ca462a382 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:14.115027 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbabb29e-2f9a-454c-b149-468ca462a382 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:14.116801 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 804/1615] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:13 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:14.117640 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce5eef-f251-47ae-b80c-8fd10e4b88dc None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:14.198123 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce5eef-f251-47ae-b80c-8fd10e4b88dc None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:14.210494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7d0b7-f7f7-41c5-b838-62f0c39d46d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:14.211263 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7d0b7-f7f7-41c5-b838-62f0c39d46d8 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:14.211549 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7d0b7-f7f7-41c5-b838-62f0c39d46d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:14.211962 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7d0b7-f7f7-41c5-b838-62f0c39d46d8 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:14.212707 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7d0b7-f7f7-41c5-b838-62f0c39d46d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:14.213932 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbabb29e-2f9a-454c-b149-468ca462a382 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:14.215171 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7d0b7-f7f7-41c5-b838-62f0c39d46d8 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:14.216054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbabb29e-2f9a-454c-b149-468ca462a382 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:14.231841 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 812/1616] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:14.233418 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbabb29e-2f9a-454c-b149-468ca462a382 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:14.244147 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbabb29e-2f9a-454c-b149-468ca462a382 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:14.245235 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 805/1617] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:14 2022] POST /identity/v3/users => generated 297 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:14.247697 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7d0b7-f7f7-41c5-b838-62f0c39d46d8 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:14.252924 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c5cb86-cb36-4106-bacd-0089f0201797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:14.253835 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5cb86-cb36-4106-bacd-0089f0201797 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:14.254199 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5cb86-cb36-4106-bacd-0089f0201797 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:14.254483 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5cb86-cb36-4106-bacd-0089f0201797 admin admin] PATH_INFO: `/v3/projects/9cc44f673423451a96d29d11834c3a29/users/878230f7682348edb7258f1dace88d55/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:14.260290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7d0b7-f7f7-41c5-b838-62f0c39d46d8 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:14.265116 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93bb64-4831-461f-bccc-2ce1e81023e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:14.266190 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93bb64-4831-461f-bccc-2ce1e81023e2 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:14.267259 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93bb64-4831-461f-bccc-2ce1e81023e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:14.268092 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93bb64-4831-461f-bccc-2ce1e81023e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:14.273312 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c5cb86-cb36-4106-bacd-0089f0201797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cc44f673423451a96d29d11834c3a29, user_id=878230f7682348edb7258f1dace88d55, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:14.276349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c5cb86-cb36-4106-bacd-0089f0201797 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:14.281115 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93bb64-4831-461f-bccc-2ce1e81023e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:14.285083 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93bb64-4831-461f-bccc-2ce1e81023e2 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:14.298268 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 806/1618] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:14.300746 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93bb64-4831-461f-bccc-2ce1e81023e2 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:14.306904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 813/1619] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:14 2022] PUT /identity/v3/projects/9cc44f673423451a96d29d11834c3a29/users/878230f7682348edb7258f1dace88d55/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:14.311797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e4e2f-fc85-42a2-9c5d-5954a4ef024f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:14.312158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e4e2f-fc85-42a2-9c5d-5954a4ef024f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:14.312835 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e4e2f-fc85-42a2-9c5d-5954a4ef024f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:14.317889 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93bb64-4831-461f-bccc-2ce1e81023e2 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:14.356479 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-ac3e4e2f-fc85-42a2-9c5d-5954a4ef024f None None] MFA Rules not processed for user `10b083e6ebaf458da7a6c52c80a4423d`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:14.358540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ac3e4e2f-fc85-42a2-9c5d-5954a4ef024f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:14.388734 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbed21b-698d-42f8-849a-198590193db0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:14.389384 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbed21b-698d-42f8-849a-198590193db0 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:14.389670 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbed21b-698d-42f8-849a-198590193db0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:14.389952 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbed21b-698d-42f8-849a-198590193db0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:14.444537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 814/1620] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:14 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:14.444537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e4e2f-fc85-42a2-9c5d-5954a4ef024f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:14.464294 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbed21b-698d-42f8-849a-198590193db0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:14.468287 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbed21b-698d-42f8-849a-198590193db0 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:14.481101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 807/1621] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:14.482036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbed21b-698d-42f8-849a-198590193db0 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:14.493879 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbed21b-698d-42f8-849a-198590193db0 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:14.500285 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290486b-25c0-4705-b5c8-9a5de50aa45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:14.501961 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290486b-25c0-4705-b5c8-9a5de50aa45c admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:14.502143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290486b-25c0-4705-b5c8-9a5de50aa45c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:14.502546 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290486b-25c0-4705-b5c8-9a5de50aa45c admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:14.503571 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290486b-25c0-4705-b5c8-9a5de50aa45c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:14.505340 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290486b-25c0-4705-b5c8-9a5de50aa45c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:14.519071 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 808/1622] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:14.520318 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290486b-25c0-4705-b5c8-9a5de50aa45c admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:14.528307 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e4e2f-fc85-42a2-9c5d-5954a4ef024f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:14.534429 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323cb83f-01ad-4285-a611-780fc10787ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:14.535047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323cb83f-01ad-4285-a611-780fc10787ee admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:14.535370 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323cb83f-01ad-4285-a611-780fc10787ee admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:14.535653 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323cb83f-01ad-4285-a611-780fc10787ee admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:14.536345 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323cb83f-01ad-4285-a611-780fc10787ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:14.538744 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323cb83f-01ad-4285-a611-780fc10787ee admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:14.539501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290486b-25c0-4705-b5c8-9a5de50aa45c admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:14.549399 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5be64d-528e-4355-86f2-0c7ea79eff8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:14.550487 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5be64d-528e-4355-86f2-0c7ea79eff8c admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:14.550770 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5be64d-528e-4355-86f2-0c7ea79eff8c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:14.551046 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5be64d-528e-4355-86f2-0c7ea79eff8c admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:14.551767 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5be64d-528e-4355-86f2-0c7ea79eff8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:14.553523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5be64d-528e-4355-86f2-0c7ea79eff8c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:14.575188 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 809/1623] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:14.576310 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5be64d-528e-4355-86f2-0c7ea79eff8c admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:14.602812 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 815/1624] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:14 2022] POST /identity/v3/projects => generated 405 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:14.604522 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5be64d-528e-4355-86f2-0c7ea79eff8c admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:14.605554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323cb83f-01ad-4285-a611-780fc10787ee admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:14.724836 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323cb83f-01ad-4285-a611-780fc10787ee admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:14.734418 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24793219-c046-4832-8030-83fd4962b7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:14.735094 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24793219-c046-4832-8030-83fd4962b7b0 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:14.735437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24793219-c046-4832-8030-83fd4962b7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:14.735676 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24793219-c046-4832-8030-83fd4962b7b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:14.736676 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24793219-c046-4832-8030-83fd4962b7b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:14.739159 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24793219-c046-4832-8030-83fd4962b7b0 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:14.742614 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a44881-19a8-40f3-babb-a51b9eeb64d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:14.743520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a44881-19a8-40f3-babb-a51b9eeb64d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:14.743793 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a44881-19a8-40f3-babb-a51b9eeb64d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:14.744065 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a44881-19a8-40f3-babb-a51b9eeb64d3 admin admin] PATH_INFO: `/v3/projects/bb01e86c8dbe4e5b986653458c907ef8/users/bd7803393aa045cb9fe3f20c02fe9a88/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:14.785416 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a44881-19a8-40f3-babb-a51b9eeb64d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb01e86c8dbe4e5b986653458c907ef8, user_id=bd7803393aa045cb9fe3f20c02fe9a88, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:14.789014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a44881-19a8-40f3-babb-a51b9eeb64d3 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:14.802381 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 810/1625] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:14 2022] POST /identity/v3/projects => generated 371 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:14.803571 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24793219-c046-4832-8030-83fd4962b7b0 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:14.815777 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 816/1626] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:14 2022] PUT /identity/v3/projects/bb01e86c8dbe4e5b986653458c907ef8/users/bd7803393aa045cb9fe3f20c02fe9a88/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:14.816901 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a44881-19a8-40f3-babb-a51b9eeb64d3 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:14.891306 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24793219-c046-4832-8030-83fd4962b7b0 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:14.935550 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a44881-19a8-40f3-babb-a51b9eeb64d3 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:14.943331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df79dc7-4047-43c5-bd2f-45071fcf4b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:14.944550 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df79dc7-4047-43c5-bd2f-45071fcf4b02 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:14.945042 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df79dc7-4047-43c5-bd2f-45071fcf4b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:14.945988 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df79dc7-4047-43c5-bd2f-45071fcf4b02 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:14.947700 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df79dc7-4047-43c5-bd2f-45071fcf4b02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:14.951663 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df79dc7-4047-43c5-bd2f-45071fcf4b02 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:15.010629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 817/1627] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:14 2022] POST /identity/v3/users => generated 309 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:15.012579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df79dc7-4047-43c5-bd2f-45071fcf4b02 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:15.035118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df79dc7-4047-43c5-bd2f-45071fcf4b02 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:15.046151 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f97f44-a46f-49f2-9b61-8e2d54282839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:15.048541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f97f44-a46f-49f2-9b61-8e2d54282839 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:15.048916 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f97f44-a46f-49f2-9b61-8e2d54282839 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:15.049694 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f97f44-a46f-49f2-9b61-8e2d54282839 admin admin] PATH_INFO: `/v3/projects/9cc44f673423451a96d29d11834c3a29/users/878230f7682348edb7258f1dace88d55/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:15.049829 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716deb9-7ffc-4bf2-878b-59c32cde6049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:15.050407 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716deb9-7ffc-4bf2-878b-59c32cde6049 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:15.051614 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716deb9-7ffc-4bf2-878b-59c32cde6049 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:15.051771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716deb9-7ffc-4bf2-878b-59c32cde6049 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:15.052478 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716deb9-7ffc-4bf2-878b-59c32cde6049 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:15.054503 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f97f44-a46f-49f2-9b61-8e2d54282839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cc44f673423451a96d29d11834c3a29, user_id=878230f7682348edb7258f1dace88d55, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:15.067909 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716deb9-7ffc-4bf2-878b-59c32cde6049 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:15.075562 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f97f44-a46f-49f2-9b61-8e2d54282839 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:15.138065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 811/1628] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:14 2022] PUT /identity/v3/projects/9cc44f673423451a96d29d11834c3a29/users/878230f7682348edb7258f1dace88d55/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:15.139295 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f97f44-a46f-49f2-9b61-8e2d54282839 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:15.154800 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 818/1629] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:15 2022] POST /identity/v3/users => generated 292 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:15.157184 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716deb9-7ffc-4bf2-878b-59c32cde6049 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:15.183406 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716deb9-7ffc-4bf2-878b-59c32cde6049 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:15.288939 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f97f44-a46f-49f2-9b61-8e2d54282839 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:15.298379 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df234b74-3139-4624-b259-e8697ae63141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:15.300959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6bf1c-3ab4-40cc-9b08-f21d1de8a013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:15.301151 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df234b74-3139-4624-b259-e8697ae63141 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:15.301344 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df234b74-3139-4624-b259-e8697ae63141 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:15.301592 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df234b74-3139-4624-b259-e8697ae63141 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:15.301716 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6bf1c-3ab4-40cc-9b08-f21d1de8a013 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:15.301911 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6bf1c-3ab4-40cc-9b08-f21d1de8a013 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:15.302218 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6bf1c-3ab4-40cc-9b08-f21d1de8a013 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:15.302963 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a6bf1c-3ab4-40cc-9b08-f21d1de8a013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:15.304421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df234b74-3139-4624-b259-e8697ae63141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:15.304734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a6bf1c-3ab4-40cc-9b08-f21d1de8a013 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:15.306483 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df234b74-3139-4624-b259-e8697ae63141 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:15.315842 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 819/1630] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:15.317563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95015c46-da44-4ecf-aef8-657f871b1922 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:15.317797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95015c46-da44-4ecf-aef8-657f871b1922 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:15.318084 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95015c46-da44-4ecf-aef8-657f871b1922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:15.335246 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 812/1631] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:15.336162 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df234b74-3139-4624-b259-e8697ae63141 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:15.353180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df234b74-3139-4624-b259-e8697ae63141 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:15.353655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-95015c46-da44-4ecf-aef8-657f871b1922 None None] MFA Rules not processed for user `878230f7682348edb7258f1dace88d55`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:15.355120 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-95015c46-da44-4ecf-aef8-657f871b1922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:15.359380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd68b432-6f7e-44b3-88a7-c5ebff56b5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:15.360300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68b432-6f7e-44b3-88a7-c5ebff56b5f5 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:15.361234 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68b432-6f7e-44b3-88a7-c5ebff56b5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:15.361512 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68b432-6f7e-44b3-88a7-c5ebff56b5f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:15.362361 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd68b432-6f7e-44b3-88a7-c5ebff56b5f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:15.364147 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd68b432-6f7e-44b3-88a7-c5ebff56b5f5 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:15.376215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 813/1632] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:15.377046 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd68b432-6f7e-44b3-88a7-c5ebff56b5f5 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:15.397557 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd68b432-6f7e-44b3-88a7-c5ebff56b5f5 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:15.403214 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e3347-eb39-4e17-9289-9c58e42f1302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:15.404263 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e3347-eb39-4e17-9289-9c58e42f1302 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:15.404552 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e3347-eb39-4e17-9289-9c58e42f1302 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:15.404830 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e3347-eb39-4e17-9289-9c58e42f1302 admin admin] PATH_INFO: `/v3/projects/388507a9b9a74759b0ec64fc043c552a/users/07a49429a0d54b95887f0aee970f3167/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:15.426270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e3347-eb39-4e17-9289-9c58e42f1302 admin admin] RBAC: Authorizing `identity:create_grant(project_id=388507a9b9a74759b0ec64fc043c552a, user_id=07a49429a0d54b95887f0aee970f3167, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:15.429346 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e3347-eb39-4e17-9289-9c58e42f1302 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:15.446442 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 814/1633] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:15 2022] PUT /identity/v3/projects/388507a9b9a74759b0ec64fc043c552a/users/07a49429a0d54b95887f0aee970f3167/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:15.447531 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e3347-eb39-4e17-9289-9c58e42f1302 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:15.464527 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 820/1634] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:15.465505 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95015c46-da44-4ecf-aef8-657f871b1922 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:15.477010 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95015c46-da44-4ecf-aef8-657f871b1922 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:15.521793 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e3347-eb39-4e17-9289-9c58e42f1302 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:15.527204 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc9683-103b-4bd2-bade-9f5c60319dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:15.528326 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc9683-103b-4bd2-bade-9f5c60319dcf admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:15.528609 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc9683-103b-4bd2-bade-9f5c60319dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:15.529067 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc9683-103b-4bd2-bade-9f5c60319dcf admin admin] PATH_INFO: `/v3/projects/bb01e86c8dbe4e5b986653458c907ef8/users/bd7803393aa045cb9fe3f20c02fe9a88/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:15.531465 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc9683-103b-4bd2-bade-9f5c60319dcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb01e86c8dbe4e5b986653458c907ef8, user_id=bd7803393aa045cb9fe3f20c02fe9a88, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:15.535247 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc9683-103b-4bd2-bade-9f5c60319dcf admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:15.549479 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7d978-4162-4c10-b0f6-4a5f0419c635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:15.550526 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7d978-4162-4c10-b0f6-4a5f0419c635 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:15.550998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7d978-4162-4c10-b0f6-4a5f0419c635 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:15.551286 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7d978-4162-4c10-b0f6-4a5f0419c635 admin admin] PATH_INFO: `/v3/projects/337cfb14003948e8b662907aa21241c7/users/de507a37973141e999bcf302923fe227/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:15.559547 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 815/1635] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:15 2022] PUT /identity/v3/projects/bb01e86c8dbe4e5b986653458c907ef8/users/bd7803393aa045cb9fe3f20c02fe9a88/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:15.560568 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc9683-103b-4bd2-bade-9f5c60319dcf admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:15.572662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec7d978-4162-4c10-b0f6-4a5f0419c635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=337cfb14003948e8b662907aa21241c7, user_id=de507a37973141e999bcf302923fe227, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:15.582042 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec7d978-4162-4c10-b0f6-4a5f0419c635 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:15.603414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 821/1636] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:15 2022] PUT /identity/v3/projects/337cfb14003948e8b662907aa21241c7/users/de507a37973141e999bcf302923fe227/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:15.604340 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7d978-4162-4c10-b0f6-4a5f0419c635 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:15.649915 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc9683-103b-4bd2-bade-9f5c60319dcf admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:15.677837 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7d978-4162-4c10-b0f6-4a5f0419c635 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:15.681346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491043d-6c2b-426e-a1be-4d6b2813220c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:15.682018 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491043d-6c2b-426e-a1be-4d6b2813220c service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:15.682355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491043d-6c2b-426e-a1be-4d6b2813220c service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:15.682638 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491043d-6c2b-426e-a1be-4d6b2813220c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:15.721955 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d66d4-4e88-4517-aa59-e4cb47e5d9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:15.724486 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d66d4-4e88-4517-aa59-e4cb47e5d9ca admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:15.724951 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d66d4-4e88-4517-aa59-e4cb47e5d9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:15.725284 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d66d4-4e88-4517-aa59-e4cb47e5d9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:15.726209 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d66d4-4e88-4517-aa59-e4cb47e5d9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:15.727914 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d66d4-4e88-4517-aa59-e4cb47e5d9ca admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:15.748092 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 816/1637] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:15.749069 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabad49-2e61-49c5-8051-b061b6e07f6f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:15.749359 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabad49-2e61-49c5-8051-b061b6e07f6f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:15.749635 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabad49-2e61-49c5-8051-b061b6e07f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:15.763642 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1491043d-6c2b-426e-a1be-4d6b2813220c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:15.765099 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1491043d-6c2b-426e-a1be-4d6b2813220c service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:15.792150 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ceabad49-2e61-49c5-8051-b061b6e07f6f None None] MFA Rules not processed for user `bd7803393aa045cb9fe3f20c02fe9a88`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:15.792623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ceabad49-2e61-49c5-8051-b061b6e07f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:15.796615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 822/1638] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:15.798873 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491043d-6c2b-426e-a1be-4d6b2813220c service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:15.816466 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491043d-6c2b-426e-a1be-4d6b2813220c service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:15.841901 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674fda4-1097-4721-85f3-b69f1e2c59e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:15.842926 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674fda4-1097-4721-85f3-b69f1e2c59e2 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:15.843412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674fda4-1097-4721-85f3-b69f1e2c59e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:15.843876 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674fda4-1097-4721-85f3-b69f1e2c59e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:15.845587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674fda4-1097-4721-85f3-b69f1e2c59e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:15.848188 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674fda4-1097-4721-85f3-b69f1e2c59e2 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:15.862160 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 823/1639] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:15.864074 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674fda4-1097-4721-85f3-b69f1e2c59e2 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:15.876949 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674fda4-1097-4721-85f3-b69f1e2c59e2 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:15.884499 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe25d8-aaf1-4e58-8128-5bdf5a9ced48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:15.885828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe25d8-aaf1-4e58-8128-5bdf5a9ced48 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:15.886696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe25d8-aaf1-4e58-8128-5bdf5a9ced48 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:15.887561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe25d8-aaf1-4e58-8128-5bdf5a9ced48 admin admin] PATH_INFO: `/v3/projects/388507a9b9a74759b0ec64fc043c552a/users/07a49429a0d54b95887f0aee970f3167/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:15.889948 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe25d8-aaf1-4e58-8128-5bdf5a9ced48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=388507a9b9a74759b0ec64fc043c552a, user_id=07a49429a0d54b95887f0aee970f3167, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:15.894500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe25d8-aaf1-4e58-8128-5bdf5a9ced48 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:15.909068 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 824/1640] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:15 2022] PUT /identity/v3/projects/388507a9b9a74759b0ec64fc043c552a/users/07a49429a0d54b95887f0aee970f3167/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:15.910921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe25d8-aaf1-4e58-8128-5bdf5a9ced48 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:15.918188 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 817/1641] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:15 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:15.920596 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e845b8a-a333-4b73-a6fa-ceec232b07b6 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:15.921064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e845b8a-a333-4b73-a6fa-ceec232b07b6 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:15.921659 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e845b8a-a333-4b73-a6fa-ceec232b07b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:15.976068 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-8e845b8a-a333-4b73-a6fa-ceec232b07b6 None None] MFA Rules not processed for user `07a49429a0d54b95887f0aee970f3167`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:15.977224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8e845b8a-a333-4b73-a6fa-ceec232b07b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:16.030064 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe25d8-aaf1-4e58-8128-5bdf5a9ced48 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:16.036018 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98063b41-2f8c-4eb7-98d8-fa7e995627c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:16.037027 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98063b41-2f8c-4eb7-98d8-fa7e995627c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:16.037240 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98063b41-2f8c-4eb7-98d8-fa7e995627c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:16.037467 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98063b41-2f8c-4eb7-98d8-fa7e995627c3 admin admin] PATH_INFO: `/v3/projects/337cfb14003948e8b662907aa21241c7/users/de507a37973141e999bcf302923fe227/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:16.039681 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98063b41-2f8c-4eb7-98d8-fa7e995627c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=337cfb14003948e8b662907aa21241c7, user_id=de507a37973141e999bcf302923fe227, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:16.042736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98063b41-2f8c-4eb7-98d8-fa7e995627c3 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:16.055241 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 825/1642] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:15 2022] PUT /identity/v3/projects/337cfb14003948e8b662907aa21241c7/users/de507a37973141e999bcf302923fe227/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:16.055898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98063b41-2f8c-4eb7-98d8-fa7e995627c3 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:16.130391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 818/1643] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:15 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:16.134746 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e845b8a-a333-4b73-a6fa-ceec232b07b6 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:16.137919 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98063b41-2f8c-4eb7-98d8-fa7e995627c3 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:16.146570 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5481271-04eb-4d78-84e5-de1c65dc7a08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:16.147731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5481271-04eb-4d78-84e5-de1c65dc7a08 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:16.148276 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5481271-04eb-4d78-84e5-de1c65dc7a08 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:16.149458 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5481271-04eb-4d78-84e5-de1c65dc7a08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:16.247520 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e845b8a-a333-4b73-a6fa-ceec232b07b6 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:16.256567 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5481271-04eb-4d78-84e5-de1c65dc7a08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:16.257665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc463057-91cc-49c5-8112-6af4f8932ffe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:16.258306 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc463057-91cc-49c5-8112-6af4f8932ffe service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:16.258589 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc463057-91cc-49c5-8112-6af4f8932ffe service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:16.258845 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc463057-91cc-49c5-8112-6af4f8932ffe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:16.260374 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5481271-04eb-4d78-84e5-de1c65dc7a08 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:16.273901 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 826/1644] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:16 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:16.275959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382875f4-23e0-4787-a230-f86f099b2519 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:16.276447 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382875f4-23e0-4787-a230-f86f099b2519 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:16.277215 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382875f4-23e0-4787-a230-f86f099b2519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:16.313756 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-382875f4-23e0-4787-a230-f86f099b2519 None None] MFA Rules not processed for user `de507a37973141e999bcf302923fe227`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:16.316090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-382875f4-23e0-4787-a230-f86f099b2519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:16.341382 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc463057-91cc-49c5-8112-6af4f8932ffe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:16.343247 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc463057-91cc-49c5-8112-6af4f8932ffe service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:16.363506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 819/1645] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:16.387134 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc463057-91cc-49c5-8112-6af4f8932ffe service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:16.414828 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc463057-91cc-49c5-8112-6af4f8932ffe service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:16.423921 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09095a0-2cf3-4777-9b78-e44fab98e825 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:16.425741 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09095a0-2cf3-4777-9b78-e44fab98e825 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:16.439538 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09095a0-2cf3-4777-9b78-e44fab98e825 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:16.439764 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09095a0-2cf3-4777-9b78-e44fab98e825 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:16.475895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 827/1646] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:16 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:16.475895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528594a-c0bc-499d-a19b-66df57323190 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:16.475895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528594a-c0bc-499d-a19b-66df57323190 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:16.477785 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528594a-c0bc-499d-a19b-66df57323190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:16.539913 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e528594a-c0bc-499d-a19b-66df57323190 None None] MFA Rules not processed for user `3dc4b31fcb154251ac27e4e4dce5e6c7`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:16.541160 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e528594a-c0bc-499d-a19b-66df57323190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:16.592323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09095a0-2cf3-4777-9b78-e44fab98e825 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:16.593994 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09095a0-2cf3-4777-9b78-e44fab98e825 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:16.608283 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 820/1647] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:16 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:16.699498 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 828/1648] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:16 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:16.704408 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09095a0-2cf3-4777-9b78-e44fab98e825 service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:16.842647 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528594a-c0bc-499d-a19b-66df57323190 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:16.854480 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528594a-c0bc-499d-a19b-66df57323190 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:16.858912 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c5815-7c75-4c8e-845f-aed6c31bb55f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:16.859689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c5815-7c75-4c8e-845f-aed6c31bb55f service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:16.860073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c5815-7c75-4c8e-845f-aed6c31bb55f service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:16.861026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c5815-7c75-4c8e-845f-aed6c31bb55f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:16.874063 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c5815-7c75-4c8e-845f-aed6c31bb55f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:16.878103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c5815-7c75-4c8e-845f-aed6c31bb55f service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:16.927245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09095a0-2cf3-4777-9b78-e44fab98e825 service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:16.932735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 829/1649] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:16 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 23:02:16.939117 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e5f6c-3823-4370-bbea-99a22e91ab98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:16.939793 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e5f6c-3823-4370-bbea-99a22e91ab98 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:16.940075 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e5f6c-3823-4370-bbea-99a22e91ab98 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:16.940344 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e5f6c-3823-4370-bbea-99a22e91ab98 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:16.941043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e5f6c-3823-4370-bbea-99a22e91ab98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:16.943755 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e5f6c-3823-4370-bbea-99a22e91ab98 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:17.017411 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 821/1650] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:16 2022] POST /identity/v3/projects => generated 405 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:17.021433 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c5815-7c75-4c8e-845f-aed6c31bb55f service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:17.049335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c5815-7c75-4c8e-845f-aed6c31bb55f service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:17.129766 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a12da-393e-437f-a4c7-af441e9627f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:17.130201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a12da-393e-437f-a4c7-af441e9627f8 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:17.130799 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a12da-393e-437f-a4c7-af441e9627f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:17.131069 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a12da-393e-437f-a4c7-af441e9627f8 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:17.132713 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a12da-393e-437f-a4c7-af441e9627f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:17.134459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a12da-393e-437f-a4c7-af441e9627f8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:17.177255 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 830/1651] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:17 2022] POST /identity/v3/users => generated 309 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:17.178674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e5f6c-3823-4370-bbea-99a22e91ab98 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:17.194318 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e5f6c-3823-4370-bbea-99a22e91ab98 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:17.210587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b43648-1949-4b69-853d-25e12c903219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:17.211365 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b43648-1949-4b69-853d-25e12c903219 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:17.211715 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b43648-1949-4b69-853d-25e12c903219 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:17.212059 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b43648-1949-4b69-853d-25e12c903219 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:17.212856 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b43648-1949-4b69-853d-25e12c903219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:17.231292 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b43648-1949-4b69-853d-25e12c903219 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:17.272708 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 822/1652] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:17.279216 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a12da-393e-437f-a4c7-af441e9627f8 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:17.318728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a12da-393e-437f-a4c7-af441e9627f8 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:17.319911 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b43648-1949-4b69-853d-25e12c903219 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:17.356266 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b43648-1949-4b69-853d-25e12c903219 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:17.356987 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacaba2-dd15-4a3a-9016-ceb92a8f368a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:17.361906 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacaba2-dd15-4a3a-9016-ceb92a8f368a admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:17.366480 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacaba2-dd15-4a3a-9016-ceb92a8f368a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:17.367282 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacaba2-dd15-4a3a-9016-ceb92a8f368a admin admin] PATH_INFO: `/v3/projects/66d64afef3c14aa19d106c06c7a1f2e2/users/f4b7863576de4dafa7695defe1e8ed64/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:17.386080 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c63727-fc21-41ee-93d2-141c6a13ba22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:17.387894 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c63727-fc21-41ee-93d2-141c6a13ba22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:17.388176 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c63727-fc21-41ee-93d2-141c6a13ba22 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:17.388454 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c63727-fc21-41ee-93d2-141c6a13ba22 admin admin] PATH_INFO: `/v3/users/461bf24c5b1a429f8f52b036fe768b5d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:17.390383 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c63727-fc21-41ee-93d2-141c6a13ba22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=461bf24c5b1a429f8f52b036fe768b5d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:17.392030 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: WARNING py.warnings [None req-52c63727-fc21-41ee-93d2-141c6a13ba22 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 23:02:17.392030 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: warnings.warn(msg) May 09 23:02:17.392030 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: May 09 23:02:17.393591 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c63727-fc21-41ee-93d2-141c6a13ba22 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:17.437183 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbacaba2-dd15-4a3a-9016-ceb92a8f368a admin admin] RBAC: Authorizing `identity:create_grant(project_id=66d64afef3c14aa19d106c06c7a1f2e2, user_id=f4b7863576de4dafa7695defe1e8ed64, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:17.444639 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbacaba2-dd15-4a3a-9016-ceb92a8f368a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:17.496132 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 831/1653] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:17 2022] PUT /identity/v3/projects/66d64afef3c14aa19d106c06c7a1f2e2/users/f4b7863576de4dafa7695defe1e8ed64/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:17.497200 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacaba2-dd15-4a3a-9016-ceb92a8f368a admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:17.564728 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-52c63727-fc21-41ee-93d2-141c6a13ba22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '461bf24c5b1a429f8f52b036fe768b5d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:17.594995 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-52c63727-fc21-41ee-93d2-141c6a13ba22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '461bf24c5b1a429f8f52b036fe768b5d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:17.596257 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-52c63727-fc21-41ee-93d2-141c6a13ba22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '461bf24c5b1a429f8f52b036fe768b5d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:17.646752 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-52c63727-fc21-41ee-93d2-141c6a13ba22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '461bf24c5b1a429f8f52b036fe768b5d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:17.648025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-52c63727-fc21-41ee-93d2-141c6a13ba22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '461bf24c5b1a429f8f52b036fe768b5d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:17.668392 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacaba2-dd15-4a3a-9016-ceb92a8f368a admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:17.685814 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2a82b-4d96-453f-b592-5fae8cf8f923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:17.689592 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2a82b-4d96-453f-b592-5fae8cf8f923 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:17.689960 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2a82b-4d96-453f-b592-5fae8cf8f923 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:17.690379 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2a82b-4d96-453f-b592-5fae8cf8f923 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:17.695344 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2a82b-4d96-453f-b592-5fae8cf8f923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:17.700863 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2a82b-4d96-453f-b592-5fae8cf8f923 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:17.739369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 832/1655] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:02:17.740389 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 823/1655] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:17 2022] DELETE /identity/v3/users/461bf24c5b1a429f8f52b036fe768b5d => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:02:17.740857 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c3b03-8a21-485f-ac24-ff99a1d6cb4a None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:17.741152 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c3b03-8a21-485f-ac24-ff99a1d6cb4a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:17.741434 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c3b03-8a21-485f-ac24-ff99a1d6cb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:17.741891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c63727-fc21-41ee-93d2-141c6a13ba22 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:17.745826 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-52c63727-fc21-41ee-93d2-141c6a13ba22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:17.832826 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-967c3b03-8a21-485f-ac24-ff99a1d6cb4a None None] MFA Rules not processed for user `90018602e07f4a4ca5f46c5c85bc818e`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:17.834066 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-967c3b03-8a21-485f-ac24-ff99a1d6cb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:17.885228 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c63727-fc21-41ee-93d2-141c6a13ba22 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:17.890257 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497481bc-71be-47fe-9b44-c431b650ddf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:17.891159 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497481bc-71be-47fe-9b44-c431b650ddf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:17.891979 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497481bc-71be-47fe-9b44-c431b650ddf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:17.892438 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497481bc-71be-47fe-9b44-c431b650ddf6 admin admin] PATH_INFO: `/v3/projects/66d64afef3c14aa19d106c06c7a1f2e2/users/f4b7863576de4dafa7695defe1e8ed64/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:17.898317 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497481bc-71be-47fe-9b44-c431b650ddf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66d64afef3c14aa19d106c06c7a1f2e2, user_id=f4b7863576de4dafa7695defe1e8ed64, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:17.901226 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497481bc-71be-47fe-9b44-c431b650ddf6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:17.919922 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 824/1656] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:17 2022] PUT /identity/v3/projects/66d64afef3c14aa19d106c06c7a1f2e2/users/f4b7863576de4dafa7695defe1e8ed64/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:17.922502 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3710293-59c2-4182-b082-e2441a9eb681 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:17.923229 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3710293-59c2-4182-b082-e2441a9eb681 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:17.923582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3710293-59c2-4182-b082-e2441a9eb681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:17.945881 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b3710293-59c2-4182-b082-e2441a9eb681 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:17.947306 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b3710293-59c2-4182-b082-e2441a9eb681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:18.027202 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 825/1657] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:18.028302 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3710293-59c2-4182-b082-e2441a9eb681 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:18.029840 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b3710293-59c2-4182-b082-e2441a9eb681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:18.035073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 833/1658] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:18.039266 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77b73e-8f40-4e23-9397-b6dcbf8c1f5a None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:18.039561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77b73e-8f40-4e23-9397-b6dcbf8c1f5a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:18.039842 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77b73e-8f40-4e23-9397-b6dcbf8c1f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:18.077286 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-4a77b73e-8f40-4e23-9397-b6dcbf8c1f5a None None] MFA Rules not processed for user `90018602e07f4a4ca5f46c5c85bc818e`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:18.080697 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4a77b73e-8f40-4e23-9397-b6dcbf8c1f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:18.120237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3710293-59c2-4182-b082-e2441a9eb681 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:18.126539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ede85-0991-4c6f-b5da-87cd0e680c4c tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7803393aa045cb9fe3f20c02fe9a88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb01e86c8dbe4e5b986653458c907ef8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:18.127306 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ede85-0991-4c6f-b5da-87cd0e680c4c tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:18.128158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ede85-0991-4c6f-b5da-87cd0e680c4c tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:18.128158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ede85-0991-4c6f-b5da-87cd0e680c4c tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:18.129231 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ede85-0991-4c6f-b5da-87cd0e680c4c tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:02:18.129538 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ede85-0991-4c6f-b5da-87cd0e680c4c tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:18.131409 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ede85-0991-4c6f-b5da-87cd0e680c4c tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:18.148562 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 826/1659] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:02:18.149235 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ede85-0991-4c6f-b5da-87cd0e680c4c tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:18.151386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c59ede85-0991-4c6f-b5da-87cd0e680c4c tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:18.214565 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 834/1660] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:18.215603 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77b73e-8f40-4e23-9397-b6dcbf8c1f5a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:18.218562 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4a77b73e-8f40-4e23-9397-b6dcbf8c1f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:18.249781 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77b73e-8f40-4e23-9397-b6dcbf8c1f5a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:18.262581 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d1f63-4dae-426e-98f5-d08165f4383e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:18.263477 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d1f63-4dae-426e-98f5-d08165f4383e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:18.263755 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d1f63-4dae-426e-98f5-d08165f4383e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:18.264072 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d1f63-4dae-426e-98f5-d08165f4383e admin admin] PATH_INFO: `/v3/users/10b083e6ebaf458da7a6c52c80a4423d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:18.267418 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d1f63-4dae-426e-98f5-d08165f4383e admin admin] RBAC: Authorizing `identity:delete_user(user_id=10b083e6ebaf458da7a6c52c80a4423d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:18.270501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d1f63-4dae-426e-98f5-d08165f4383e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:18.285542 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ede85-0991-4c6f-b5da-87cd0e680c4c tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:18.298396 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb3588-2093-4de4-a69f-8f4f3eb14343 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:18.299103 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb3588-2093-4de4-a69f-8f4f3eb14343 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:18.299372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb3588-2093-4de4-a69f-8f4f3eb14343 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:18.299650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb3588-2093-4de4-a69f-8f4f3eb14343 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:18.315392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb3588-2093-4de4-a69f-8f4f3eb14343 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:18.316860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb3588-2093-4de4-a69f-8f4f3eb14343 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:18.342381 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 827/1661] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:18.343055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb3588-2093-4de4-a69f-8f4f3eb14343 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:18.365241 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb3588-2093-4de4-a69f-8f4f3eb14343 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:18.392606 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4724e-7487-4eec-aa40-a274efe96a61 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7803393aa045cb9fe3f20c02fe9a88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb01e86c8dbe4e5b986653458c907ef8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:18.393283 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4724e-7487-4eec-aa40-a274efe96a61 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:18.393571 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4724e-7487-4eec-aa40-a274efe96a61 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:18.393850 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4724e-7487-4eec-aa40-a274efe96a61 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:18.394596 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d4724e-7487-4eec-aa40-a274efe96a61 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:18.396673 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d4724e-7487-4eec-aa40-a274efe96a61 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:18.396833 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-5a8d1f63-4dae-426e-98f5-d08165f4383e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10b083e6ebaf458da7a6c52c80a4423d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:18.413261 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-5a8d1f63-4dae-426e-98f5-d08165f4383e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10b083e6ebaf458da7a6c52c80a4423d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:18.448060 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-5a8d1f63-4dae-426e-98f5-d08165f4383e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10b083e6ebaf458da7a6c52c80a4423d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:18.448771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-5a8d1f63-4dae-426e-98f5-d08165f4383e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10b083e6ebaf458da7a6c52c80a4423d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:18.462719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 828/1662] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:18 2022] POST /identity/v3/projects => generated 355 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:18.464657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f725557-5e9e-4550-a2c4-b42816442f27 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:18.475053 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-5a8d1f63-4dae-426e-98f5-d08165f4383e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10b083e6ebaf458da7a6c52c80a4423d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:18.476567 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f725557-5e9e-4550-a2c4-b42816442f27 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:18.477105 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f725557-5e9e-4550-a2c4-b42816442f27 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:18.478940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 829/1663] 10.222.0.23 () {54 vars in 961 bytes} [Mon May 9 23:02:18 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 23:02:18.479966 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f725557-5e9e-4550-a2c4-b42816442f27 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:18.489944 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7f725557-5e9e-4550-a2c4-b42816442f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:18.496288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 835/1664] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:18 2022] DELETE /identity/v3/users/10b083e6ebaf458da7a6c52c80a4423d => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:18.497922 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b5fb5-2c60-4d7c-880b-f15b7218e6d0 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:18.499326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b5fb5-2c60-4d7c-880b-f15b7218e6d0 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:18.499651 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b5fb5-2c60-4d7c-880b-f15b7218e6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:18.555226 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-666b5fb5-2c60-4d7c-880b-f15b7218e6d0 None None] MFA Rules not processed for user `02fcd7cc86134ce488c7dd75dfb174b8`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:18.559251 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-666b5fb5-2c60-4d7c-880b-f15b7218e6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:18.667453 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f725557-5e9e-4550-a2c4-b42816442f27 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:18.678490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276baf7-f631-40b2-ad5b-fad2a83e8da4 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7803393aa045cb9fe3f20c02fe9a88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb01e86c8dbe4e5b986653458c907ef8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:18.679572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276baf7-f631-40b2-ad5b-fad2a83e8da4 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:18.679787 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276baf7-f631-40b2-ad5b-fad2a83e8da4 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:18.680041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276baf7-f631-40b2-ad5b-fad2a83e8da4 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:18.680998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7276baf7-f631-40b2-ad5b-fad2a83e8da4 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:02:18.681310 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7276baf7-f631-40b2-ad5b-fad2a83e8da4 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:18.683868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7276baf7-f631-40b2-ad5b-fad2a83e8da4 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:18.705531 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 830/1665] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:02:18.714110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48990be9-1f8d-4757-aed0-61be385c719b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:18.714433 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48990be9-1f8d-4757-aed0-61be385c719b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:18.714708 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48990be9-1f8d-4757-aed0-61be385c719b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:18.741747 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 836/1666] 10.222.0.23 () {58 vars in 1058 bytes} [Mon May 9 23:02:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:18.742789 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b5fb5-2c60-4d7c-880b-f15b7218e6d0 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:18.759195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-48990be9-1f8d-4757-aed0-61be385c719b None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:18.762856 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-48990be9-1f8d-4757-aed0-61be385c719b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:18.766088 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b5fb5-2c60-4d7c-880b-f15b7218e6d0 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:18.782585 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4016af-68c3-4e75-a7e0-1ee1fa504c29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:18.783332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4016af-68c3-4e75-a7e0-1ee1fa504c29 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:18.785454 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4016af-68c3-4e75-a7e0-1ee1fa504c29 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:18.785973 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4016af-68c3-4e75-a7e0-1ee1fa504c29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:18.792103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0e4016af-68c3-4e75-a7e0-1ee1fa504c29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:18.889410 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4016af-68c3-4e75-a7e0-1ee1fa504c29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:18.891698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4016af-68c3-4e75-a7e0-1ee1fa504c29 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:18.908765 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 837/1667] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:18.909979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4016af-68c3-4e75-a7e0-1ee1fa504c29 service nova] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:18.916870 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 831/1668] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:18.927687 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4016af-68c3-4e75-a7e0-1ee1fa504c29 service nova] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:18.936063 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1d3dc-f968-406e-bf56-c30aa79d9b65 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7803393aa045cb9fe3f20c02fe9a88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb01e86c8dbe4e5b986653458c907ef8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:18.937201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1d3dc-f968-406e-bf56-c30aa79d9b65 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:18.937972 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1d3dc-f968-406e-bf56-c30aa79d9b65 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:18.939354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1d3dc-f968-406e-bf56-c30aa79d9b65 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] PATH_INFO: `/v3/projects/1d00ca01c7a747d98e650ec82ff99228` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:18.941013 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1d3dc-f968-406e-bf56-c30aa79d9b65 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1d00ca01c7a747d98e650ec82ff99228)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:18.946470 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: WARNING py.warnings [None req-7bb1d3dc-f968-406e-bf56-c30aa79d9b65 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 23:02:18.946470 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: warnings.warn(msg) May 09 23:02:18.946470 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: May 09 23:02:18.950651 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1d3dc-f968-406e-bf56-c30aa79d9b65 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:18.970443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48990be9-1f8d-4757-aed0-61be385c719b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:18.972363 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-48990be9-1f8d-4757-aed0-61be385c719b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:19.095949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48990be9-1f8d-4757-aed0-61be385c719b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:19.117940 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7bb1d3dc-f968-406e-bf56-c30aa79d9b65 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d00ca01c7a747d98e650ec82ff99228', 'request_id': 'req-7bb1d3dc-f968-406e-bf56-c30aa79d9b65'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:19.131972 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7bb1d3dc-f968-406e-bf56-c30aa79d9b65 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d00ca01c7a747d98e650ec82ff99228', 'request_id': 'req-7bb1d3dc-f968-406e-bf56-c30aa79d9b65'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:19.166829 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7bb1d3dc-f968-406e-bf56-c30aa79d9b65 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d00ca01c7a747d98e650ec82ff99228', 'request_id': 'req-7bb1d3dc-f968-406e-bf56-c30aa79d9b65'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:19.194292 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d1df3-17b7-4468-be2d-d04e5fee837d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:19.195069 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d1df3-17b7-4468-be2d-d04e5fee837d service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:19.195448 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d1df3-17b7-4468-be2d-d04e5fee837d service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:19.195828 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d1df3-17b7-4468-be2d-d04e5fee837d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:19.201413 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7bb1d3dc-f968-406e-bf56-c30aa79d9b65 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] The token cache is being invalidate because project 1d00ca01c7a747d98e650ec82ff99228 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:19.201709 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7bb1d3dc-f968-406e-bf56-c30aa79d9b65 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:19.202744 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7bb1d3dc-f968-406e-bf56-c30aa79d9b65 tempest-VolumeQuotaClassesTest-1885694311 tempest-VolumeQuotaClassesTest-1885694311-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:19.207578 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 838/1669] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:18 2022] DELETE /identity/v3/projects/1d00ca01c7a747d98e650ec82ff99228 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:19.319303 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d1df3-17b7-4468-be2d-d04e5fee837d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:19.321039 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d1df3-17b7-4468-be2d-d04e5fee837d service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:19.342285 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 832/1670] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 23:02:19.468524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfabae-3f75-485a-83fa-b1ea64d647fe None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:19.468810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfabae-3f75-485a-83fa-b1ea64d647fe None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:19.469091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfabae-3f75-485a-83fa-b1ea64d647fe None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:19.471990 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 839/1671] 10.222.0.23 () {54 vars in 912 bytes} [Mon May 9 23:02:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 23:02:19.478546 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892d8b3-c1d2-46fa-9476-0e16e6564b01 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:19.478859 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892d8b3-c1d2-46fa-9476-0e16e6564b01 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:19.479138 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892d8b3-c1d2-46fa-9476-0e16e6564b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:19.521484 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-1892d8b3-c1d2-46fa-9476-0e16e6564b01 None None] MFA Rules not processed for user `02fcd7cc86134ce488c7dd75dfb174b8`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:19.523010 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1892d8b3-c1d2-46fa-9476-0e16e6564b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:19.707350 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cfabae-3f75-485a-83fa-b1ea64d647fe None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:19.709118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-01cfabae-3f75-485a-83fa-b1ea64d647fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:19.755581 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cfabae-3f75-485a-83fa-b1ea64d647fe None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:19.762014 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 833/1672] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 9 23:02:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:19.771586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d43d6c-7df0-40fc-a75e-7b31d04d66d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:19.772065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892d8b3-c1d2-46fa-9476-0e16e6564b01 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:19.772496 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d43d6c-7df0-40fc-a75e-7b31d04d66d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:19.772767 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d43d6c-7df0-40fc-a75e-7b31d04d66d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:19.773042 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d43d6c-7df0-40fc-a75e-7b31d04d66d4 admin admin] PATH_INFO: `/v3/projects/b398b8a010f548a28e72c7dd20789571` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:19.773931 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1892d8b3-c1d2-46fa-9476-0e16e6564b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:19.780174 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d43d6c-7df0-40fc-a75e-7b31d04d66d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b398b8a010f548a28e72c7dd20789571)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:19.782202 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d43d6c-7df0-40fc-a75e-7b31d04d66d4 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:19.902954 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892d8b3-c1d2-46fa-9476-0e16e6564b01 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:19.951241 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f9d43d6c-7df0-40fc-a75e-7b31d04d66d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b398b8a010f548a28e72c7dd20789571', 'request_id': 'req-f9d43d6c-7df0-40fc-a75e-7b31d04d66d4'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:19.957867 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f9d43d6c-7df0-40fc-a75e-7b31d04d66d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b398b8a010f548a28e72c7dd20789571', 'request_id': 'req-f9d43d6c-7df0-40fc-a75e-7b31d04d66d4'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:19.979239 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f9d43d6c-7df0-40fc-a75e-7b31d04d66d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b398b8a010f548a28e72c7dd20789571', 'request_id': 'req-f9d43d6c-7df0-40fc-a75e-7b31d04d66d4'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:19.997884 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f9d43d6c-7df0-40fc-a75e-7b31d04d66d4 admin admin] The token cache is being invalidate because project b398b8a010f548a28e72c7dd20789571 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:19.998220 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f9d43d6c-7df0-40fc-a75e-7b31d04d66d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:20.003896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f9d43d6c-7df0-40fc-a75e-7b31d04d66d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:20.004194 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fa41c-dbee-47cc-aa70-d4d15f054e33 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81caf0558f2449cbbc8f4cd9446f363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:20.004993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fa41c-dbee-47cc-aa70-d4d15f054e33 service glance] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:20.005276 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fa41c-dbee-47cc-aa70-d4d15f054e33 service glance] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:20.005549 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fa41c-dbee-47cc-aa70-d4d15f054e33 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:20.005901 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 840/1673] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:19 2022] DELETE /identity/v3/projects/b398b8a010f548a28e72c7dd20789571 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:20.006843 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d43d6c-7df0-40fc-a75e-7b31d04d66d4 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:20.010282 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f9d43d6c-7df0-40fc-a75e-7b31d04d66d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:20.011668 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-006fa41c-dbee-47cc-aa70-d4d15f054e33 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:20.152098 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d43d6c-7df0-40fc-a75e-7b31d04d66d4 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:20.158742 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6d2b5-559b-44b3-911d-d36e85d81f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:20.159793 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6d2b5-559b-44b3-911d-d36e85d81f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:20.160078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6d2b5-559b-44b3-911d-d36e85d81f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:20.160410 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6d2b5-559b-44b3-911d-d36e85d81f46 admin admin] PATH_INFO: `/v3/users/58586c1f929f4218abdf6862a092f83d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:20.162051 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6d2b5-559b-44b3-911d-d36e85d81f46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58586c1f929f4218abdf6862a092f83d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:20.163971 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6d2b5-559b-44b3-911d-d36e85d81f46 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:20.261246 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006fa41c-dbee-47cc-aa70-d4d15f054e33 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:20.263448 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-63c6d2b5-559b-44b3-911d-d36e85d81f46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58586c1f929f4218abdf6862a092f83d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:20.263564 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006fa41c-dbee-47cc-aa70-d4d15f054e33 service glance] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:20.280417 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-63c6d2b5-559b-44b3-911d-d36e85d81f46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58586c1f929f4218abdf6862a092f83d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:20.296940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 834/1674] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:20.300337 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fa41c-dbee-47cc-aa70-d4d15f054e33 service glance] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:20.302998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-63c6d2b5-559b-44b3-911d-d36e85d81f46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58586c1f929f4218abdf6862a092f83d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:20.303881 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-006fa41c-dbee-47cc-aa70-d4d15f054e33 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:20.305192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-63c6d2b5-559b-44b3-911d-d36e85d81f46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58586c1f929f4218abdf6862a092f83d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:20.349293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-63c6d2b5-559b-44b3-911d-d36e85d81f46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58586c1f929f4218abdf6862a092f83d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:20.354379 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fa41c-dbee-47cc-aa70-d4d15f054e33 service glance] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:20.362152 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0713456-710f-4edd-a906-7ac388bb9ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:20.362562 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 841/1675] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:20 2022] DELETE /identity/v3/users/58586c1f929f4218abdf6862a092f83d => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:20.363396 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0713456-710f-4edd-a906-7ac388bb9ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:20.363686 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0713456-710f-4edd-a906-7ac388bb9ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:20.363961 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0713456-710f-4edd-a906-7ac388bb9ea9 admin admin] PATH_INFO: `/v3/users/07a49429a0d54b95887f0aee970f3167` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:20.366428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0713456-710f-4edd-a906-7ac388bb9ea9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07a49429a0d54b95887f0aee970f3167)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:20.368387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0713456-710f-4edd-a906-7ac388bb9ea9 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:20.374153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a84c55-3f44-4a6d-b8a5-0cea1aba0442 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:20.377292 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a84c55-3f44-4a6d-b8a5-0cea1aba0442 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:20.377657 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a84c55-3f44-4a6d-b8a5-0cea1aba0442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:20.431525 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f8a84c55-3f44-4a6d-b8a5-0cea1aba0442 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:20.432852 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f8a84c55-3f44-4a6d-b8a5-0cea1aba0442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:20.469270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f0713456-710f-4edd-a906-7ac388bb9ea9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07a49429a0d54b95887f0aee970f3167'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:20.487725 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f0713456-710f-4edd-a906-7ac388bb9ea9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07a49429a0d54b95887f0aee970f3167'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:20.492416 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f0713456-710f-4edd-a906-7ac388bb9ea9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07a49429a0d54b95887f0aee970f3167'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:20.526893 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f0713456-710f-4edd-a906-7ac388bb9ea9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07a49429a0d54b95887f0aee970f3167'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:20.527971 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f0713456-710f-4edd-a906-7ac388bb9ea9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07a49429a0d54b95887f0aee970f3167'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:20.581825 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 835/1676] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:20 2022] DELETE /identity/v3/users/07a49429a0d54b95887f0aee970f3167 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:20.710342 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 842/1677] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:20.870314 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0713456-710f-4edd-a906-7ac388bb9ea9 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:20.872158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f0713456-710f-4edd-a906-7ac388bb9ea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:21.056967 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0713456-710f-4edd-a906-7ac388bb9ea9 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:21.072336 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9819cb-0968-43cf-a32f-bcac2c0b1377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:21.074023 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9819cb-0968-43cf-a32f-bcac2c0b1377 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:21.074481 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9819cb-0968-43cf-a32f-bcac2c0b1377 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:21.074899 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9819cb-0968-43cf-a32f-bcac2c0b1377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:21.262938 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9819cb-0968-43cf-a32f-bcac2c0b1377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:21.265437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9819cb-0968-43cf-a32f-bcac2c0b1377 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:21.279377 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 836/1678] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:21.589769 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a84c55-3f44-4a6d-b8a5-0cea1aba0442 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:21.594891 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f8a84c55-3f44-4a6d-b8a5-0cea1aba0442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:21.600945 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddda3e3-32f0-4b32-9752-6742d341adfe None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:21.601990 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddda3e3-32f0-4b32-9752-6742d341adfe None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:21.602427 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddda3e3-32f0-4b32-9752-6742d341adfe None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:21.603959 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 837/1679] 10.222.0.23 () {54 vars in 912 bytes} [Mon May 9 23:02:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 23:02:21.611631 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b71a73-b648-40c8-b767-a86140cf2362 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:21.612110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b71a73-b648-40c8-b767-a86140cf2362 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:21.612525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b71a73-b648-40c8-b767-a86140cf2362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:21.636361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a84c55-3f44-4a6d-b8a5-0cea1aba0442 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:21.667979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399c770-0a7e-42a9-8179-7802d405a20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:21.668929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399c770-0a7e-42a9-8179-7802d405a20f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:21.669209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399c770-0a7e-42a9-8179-7802d405a20f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:21.669486 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399c770-0a7e-42a9-8179-7802d405a20f admin admin] PATH_INFO: `/v3/projects/ee43ae34b4cd499c98fa333ee26bb26d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:21.675797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399c770-0a7e-42a9-8179-7802d405a20f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee43ae34b4cd499c98fa333ee26bb26d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:21.677739 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399c770-0a7e-42a9-8179-7802d405a20f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:21.743108 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-f4b71a73-b648-40c8-b767-a86140cf2362 None None] MFA Rules not processed for user `96fd3f6dd9e74ba7a4ad159e43e8ae96`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:21.746162 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f4b71a73-b648-40c8-b767-a86140cf2362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:21.768610 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 838/1680] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 9 23:02:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) May 09 23:02:21.803481 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b71a73-b648-40c8-b767-a86140cf2362 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:21.805490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f4b71a73-b648-40c8-b767-a86140cf2362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:21.820781 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b71a73-b648-40c8-b767-a86140cf2362 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:21.844746 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f399c770-0a7e-42a9-8179-7802d405a20f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee43ae34b4cd499c98fa333ee26bb26d', 'request_id': 'req-f399c770-0a7e-42a9-8179-7802d405a20f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:21.865264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f399c770-0a7e-42a9-8179-7802d405a20f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee43ae34b4cd499c98fa333ee26bb26d', 'request_id': 'req-f399c770-0a7e-42a9-8179-7802d405a20f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:21.876755 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f399c770-0a7e-42a9-8179-7802d405a20f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee43ae34b4cd499c98fa333ee26bb26d', 'request_id': 'req-f399c770-0a7e-42a9-8179-7802d405a20f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:21.899838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f399c770-0a7e-42a9-8179-7802d405a20f admin admin] The token cache is being invalidate because project ee43ae34b4cd499c98fa333ee26bb26d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:21.900526 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f399c770-0a7e-42a9-8179-7802d405a20f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:21.902305 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f399c770-0a7e-42a9-8179-7802d405a20f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:21.907012 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 843/1681] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:21 2022] DELETE /identity/v3/projects/ee43ae34b4cd499c98fa333ee26bb26d => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:21.909496 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0b6fc-6768-4ff7-9b33-0595d7ae8216 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:21.909943 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0b6fc-6768-4ff7-9b33-0595d7ae8216 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:21.910386 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0b6fc-6768-4ff7-9b33-0595d7ae8216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:21.931427 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17af76-564f-4ebf-803a-788bbc2b123c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:21.932103 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17af76-564f-4ebf-803a-788bbc2b123c service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:21.932433 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17af76-564f-4ebf-803a-788bbc2b123c service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:21.933688 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17af76-564f-4ebf-803a-788bbc2b123c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:21.944514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7e17af76-564f-4ebf-803a-788bbc2b123c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:21.945647 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5cb0b6fc-6768-4ff7-9b33-0595d7ae8216 None None] MFA Rules not processed for user `f4b7863576de4dafa7695defe1e8ed64`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:21.947041 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5cb0b6fc-6768-4ff7-9b33-0595d7ae8216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:21.970354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e17af76-564f-4ebf-803a-788bbc2b123c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:21.971800 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e17af76-564f-4ebf-803a-788bbc2b123c service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:21.988144 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 839/1682] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:21.988451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17af76-564f-4ebf-803a-788bbc2b123c service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:21.992239 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7e17af76-564f-4ebf-803a-788bbc2b123c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:22.142083 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 844/1683] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:21 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:22.150965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0b6fc-6768-4ff7-9b33-0595d7ae8216 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:22.153020 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5cb0b6fc-6768-4ff7-9b33-0595d7ae8216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:22.173768 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17af76-564f-4ebf-803a-788bbc2b123c service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:22.181460 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef1a9a-d608-4254-a76d-d67a406f78db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:22.183031 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef1a9a-d608-4254-a76d-d67a406f78db admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:22.183437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef1a9a-d608-4254-a76d-d67a406f78db admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:22.183980 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef1a9a-d608-4254-a76d-d67a406f78db admin admin] PATH_INFO: `/v3/users/de507a37973141e999bcf302923fe227` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:22.185717 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef1a9a-d608-4254-a76d-d67a406f78db admin admin] RBAC: Authorizing `identity:delete_user(user_id=de507a37973141e999bcf302923fe227)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:22.185971 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0b6fc-6768-4ff7-9b33-0595d7ae8216 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:22.189080 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef1a9a-d608-4254-a76d-d67a406f78db admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:22.194573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2b186-0655-4dbd-afb3-a3d64f946a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:22.195541 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2b186-0655-4dbd-afb3-a3d64f946a0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:22.196033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2b186-0655-4dbd-afb3-a3d64f946a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:22.196377 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2b186-0655-4dbd-afb3-a3d64f946a0e admin admin] PATH_INFO: `/v3/projects/388507a9b9a74759b0ec64fc043c552a` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:22.200219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f2b186-0655-4dbd-afb3-a3d64f946a0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=388507a9b9a74759b0ec64fc043c552a)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:22.203535 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f2b186-0655-4dbd-afb3-a3d64f946a0e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:22.280759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-18f2b186-0655-4dbd-afb3-a3d64f946a0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '388507a9b9a74759b0ec64fc043c552a', 'request_id': 'req-18f2b186-0655-4dbd-afb3-a3d64f946a0e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:22.283969 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-90ef1a9a-d608-4254-a76d-d67a406f78db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de507a37973141e999bcf302923fe227'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:22.295150 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-18f2b186-0655-4dbd-afb3-a3d64f946a0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '388507a9b9a74759b0ec64fc043c552a', 'request_id': 'req-18f2b186-0655-4dbd-afb3-a3d64f946a0e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:22.298933 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-90ef1a9a-d608-4254-a76d-d67a406f78db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de507a37973141e999bcf302923fe227'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:22.301801 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-90ef1a9a-d608-4254-a76d-d67a406f78db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de507a37973141e999bcf302923fe227'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:22.304386 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-18f2b186-0655-4dbd-afb3-a3d64f946a0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '388507a9b9a74759b0ec64fc043c552a', 'request_id': 'req-18f2b186-0655-4dbd-afb3-a3d64f946a0e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:22.321527 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-18f2b186-0655-4dbd-afb3-a3d64f946a0e admin admin] The token cache is being invalidate because project 388507a9b9a74759b0ec64fc043c552a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:22.321527 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-18f2b186-0655-4dbd-afb3-a3d64f946a0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:22.322311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-18f2b186-0655-4dbd-afb3-a3d64f946a0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:22.322698 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-90ef1a9a-d608-4254-a76d-d67a406f78db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de507a37973141e999bcf302923fe227'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:22.323645 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-90ef1a9a-d608-4254-a76d-d67a406f78db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de507a37973141e999bcf302923fe227'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:22.326542 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 845/1684] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:22 2022] DELETE /identity/v3/projects/388507a9b9a74759b0ec64fc043c552a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:22.334526 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7655d00-4691-43cc-97ed-6948c2971a4e None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:22.335010 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7655d00-4691-43cc-97ed-6948c2971a4e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:22.335292 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7655d00-4691-43cc-97ed-6948c2971a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:22.360067 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e7655d00-4691-43cc-97ed-6948c2971a4e None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:22.362179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e7655d00-4691-43cc-97ed-6948c2971a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:22.366587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 840/1685] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:21 2022] DELETE /identity/v3/users/de507a37973141e999bcf302923fe227 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:22.443829 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 846/1686] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:22.452325 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdea5e-9492-4022-9dc5-ea111115b9be None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:22.452907 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdea5e-9492-4022-9dc5-ea111115b9be None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:22.453454 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdea5e-9492-4022-9dc5-ea111115b9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:22.489650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-f8bdea5e-9492-4022-9dc5-ea111115b9be None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:22.490992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f8bdea5e-9492-4022-9dc5-ea111115b9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:22.509382 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 841/1687] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:22.515006 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7655d00-4691-43cc-97ed-6948c2971a4e None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:22.526229 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7655d00-4691-43cc-97ed-6948c2971a4e None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:22.532518 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5563a65-d2db-41a9-beba-e34ae33ff0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:22.533195 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5563a65-d2db-41a9-beba-e34ae33ff0cc admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:22.533477 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5563a65-d2db-41a9-beba-e34ae33ff0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:22.533753 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5563a65-d2db-41a9-beba-e34ae33ff0cc admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:22.534578 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5563a65-d2db-41a9-beba-e34ae33ff0cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:02:22.535293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5563a65-d2db-41a9-beba-e34ae33ff0cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:22.537133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5563a65-d2db-41a9-beba-e34ae33ff0cc admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:22.561497 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 847/1688] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:02:22.562085 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bdea5e-9492-4022-9dc5-ea111115b9be None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:22.577531 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bdea5e-9492-4022-9dc5-ea111115b9be None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:22.590435 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf8d15-de00-434e-b391-c376ea93a360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:22.591455 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf8d15-de00-434e-b391-c376ea93a360 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:22.591922 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf8d15-de00-434e-b391-c376ea93a360 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:22.592528 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf8d15-de00-434e-b391-c376ea93a360 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:22.593581 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf8d15-de00-434e-b391-c376ea93a360 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:22.598231 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf8d15-de00-434e-b391-c376ea93a360 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:22.677667 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5563a65-d2db-41a9-beba-e34ae33ff0cc admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:22.689926 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 842/1689] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:22 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:22.693606 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5563a65-d2db-41a9-beba-e34ae33ff0cc admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:22.744973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf8d15-de00-434e-b391-c376ea93a360 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:22.747981 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-33cf8d15-de00-434e-b391-c376ea93a360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:22.918251 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e72066-61b9-4053-9c95-4ddde6f9432a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:22.918977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e72066-61b9-4053-9c95-4ddde6f9432a admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:22.919212 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e72066-61b9-4053-9c95-4ddde6f9432a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:22.919492 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e72066-61b9-4053-9c95-4ddde6f9432a admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:22.929673 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e72066-61b9-4053-9c95-4ddde6f9432a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:22.931828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e72066-61b9-4053-9c95-4ddde6f9432a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:22.996539 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 848/1690] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:22 2022] POST /identity/v3/users => generated 294 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:23.001746 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e72066-61b9-4053-9c95-4ddde6f9432a admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:23.018766 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf8d15-de00-434e-b391-c376ea93a360 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:23.027232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e72066-61b9-4053-9c95-4ddde6f9432a admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:23.038398 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcf3af-feb0-48aa-86d8-abf2863f7588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:23.039279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcf3af-feb0-48aa-86d8-abf2863f7588 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:23.039561 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcf3af-feb0-48aa-86d8-abf2863f7588 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:23.039832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcf3af-feb0-48aa-86d8-abf2863f7588 admin admin] PATH_INFO: `/v3/projects/192f40de826942b1994a0e8ea5db9d88` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:23.044541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcf3af-feb0-48aa-86d8-abf2863f7588 admin admin] RBAC: Authorizing `identity:delete_project(project_id=192f40de826942b1994a0e8ea5db9d88)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:23.047282 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcf3af-feb0-48aa-86d8-abf2863f7588 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:23.054331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08054a4e-57ca-492e-8ab5-ca6e3b723aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:23.055021 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08054a4e-57ca-492e-8ab5-ca6e3b723aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:23.055304 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08054a4e-57ca-492e-8ab5-ca6e3b723aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:23.055580 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08054a4e-57ca-492e-8ab5-ca6e3b723aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:23.056309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08054a4e-57ca-492e-8ab5-ca6e3b723aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:23.065260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08054a4e-57ca-492e-8ab5-ca6e3b723aa2 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:23.084156 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 849/1691] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:23.085377 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89d186-4753-4c35-bebd-fe6b16c7c8dc None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:23.085668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89d186-4753-4c35-bebd-fe6b16c7c8dc None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:23.085951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89d186-4753-4c35-bebd-fe6b16c7c8dc None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:23.087271 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 850/1692] 10.222.0.23 () {54 vars in 960 bytes} [Mon May 9 23:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 23:02:23.088230 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89d186-4753-4c35-bebd-fe6b16c7c8dc None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:23.125117 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89d186-4753-4c35-bebd-fe6b16c7c8dc None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:23.161571 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-addcf3af-feb0-48aa-86d8-abf2863f7588 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '192f40de826942b1994a0e8ea5db9d88', 'request_id': 'req-addcf3af-feb0-48aa-86d8-abf2863f7588'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:23.185294 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-addcf3af-feb0-48aa-86d8-abf2863f7588 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '192f40de826942b1994a0e8ea5db9d88', 'request_id': 'req-addcf3af-feb0-48aa-86d8-abf2863f7588'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:23.208222 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-addcf3af-feb0-48aa-86d8-abf2863f7588 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '192f40de826942b1994a0e8ea5db9d88', 'request_id': 'req-addcf3af-feb0-48aa-86d8-abf2863f7588'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:23.269473 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-addcf3af-feb0-48aa-86d8-abf2863f7588 admin admin] The token cache is being invalidate because project 192f40de826942b1994a0e8ea5db9d88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:23.270370 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-addcf3af-feb0-48aa-86d8-abf2863f7588 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:23.274895 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-addcf3af-feb0-48aa-86d8-abf2863f7588 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:23.278986 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3f15e-7432-47ae-bd69-4cfbecb8dc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:23.278986 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3f15e-7432-47ae-bd69-4cfbecb8dc01 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:23.278986 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3f15e-7432-47ae-bd69-4cfbecb8dc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:23.278986 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3f15e-7432-47ae-bd69-4cfbecb8dc01 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:23.279513 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c3f15e-7432-47ae-bd69-4cfbecb8dc01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:23.280953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c3f15e-7432-47ae-bd69-4cfbecb8dc01 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:23.285271 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 843/1693] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:22 2022] DELETE /identity/v3/projects/192f40de826942b1994a0e8ea5db9d88 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:23.285271 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4acc71-930f-4eec-82e1-a63b0c79eb8c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:23.285271 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4acc71-930f-4eec-82e1-a63b0c79eb8c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:23.285271 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4acc71-930f-4eec-82e1-a63b0c79eb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:23.319890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 851/1694] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:23.321060 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3f15e-7432-47ae-bd69-4cfbecb8dc01 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:23.328415 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-83c3f15e-7432-47ae-bd69-4cfbecb8dc01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:23.354695 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3f15e-7432-47ae-bd69-4cfbecb8dc01 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:23.355500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-fe4acc71-930f-4eec-82e1-a63b0c79eb8c None None] MFA Rules not processed for user `cf9bc2fa959d4706b9f9e87509829059`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:23.356946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-fe4acc71-930f-4eec-82e1-a63b0c79eb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:23.370409 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c30d8c-ad26-4a68-88f5-5e00f52533db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:23.371319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c30d8c-ad26-4a68-88f5-5e00f52533db admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:23.371733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c30d8c-ad26-4a68-88f5-5e00f52533db admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:23.371903 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c30d8c-ad26-4a68-88f5-5e00f52533db admin admin] PATH_INFO: `/v3/projects/337cfb14003948e8b662907aa21241c7` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:23.373321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c30d8c-ad26-4a68-88f5-5e00f52533db admin admin] RBAC: Authorizing `identity:delete_project(project_id=337cfb14003948e8b662907aa21241c7)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:23.375285 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c30d8c-ad26-4a68-88f5-5e00f52533db admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:23.483734 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 844/1695] 10.222.0.23 () {58 vars in 1057 bytes} [Mon May 9 23:02:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:23.484776 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4acc71-930f-4eec-82e1-a63b0c79eb8c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:23.491762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-fe4acc71-930f-4eec-82e1-a63b0c79eb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:23.496199 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c6c30d8c-ad26-4a68-88f5-5e00f52533db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '337cfb14003948e8b662907aa21241c7', 'request_id': 'req-c6c30d8c-ad26-4a68-88f5-5e00f52533db'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:23.532299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c6c30d8c-ad26-4a68-88f5-5e00f52533db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '337cfb14003948e8b662907aa21241c7', 'request_id': 'req-c6c30d8c-ad26-4a68-88f5-5e00f52533db'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:23.551093 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4acc71-930f-4eec-82e1-a63b0c79eb8c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:23.552291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c6c30d8c-ad26-4a68-88f5-5e00f52533db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '337cfb14003948e8b662907aa21241c7', 'request_id': 'req-c6c30d8c-ad26-4a68-88f5-5e00f52533db'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:23.579589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c6c30d8c-ad26-4a68-88f5-5e00f52533db admin admin] The token cache is being invalidate because project 337cfb14003948e8b662907aa21241c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:23.579589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c6c30d8c-ad26-4a68-88f5-5e00f52533db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:23.582936 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c6c30d8c-ad26-4a68-88f5-5e00f52533db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:23.584940 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 852/1696] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:23 2022] DELETE /identity/v3/projects/337cfb14003948e8b662907aa21241c7 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:23.585715 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c30d8c-ad26-4a68-88f5-5e00f52533db admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:23.589132 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c6c30d8c-ad26-4a68-88f5-5e00f52533db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:23.666815 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c867b3-8eaf-4bb6-9901-f12d9155ec34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:23.668141 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c867b3-8eaf-4bb6-9901-f12d9155ec34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:23.668730 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c867b3-8eaf-4bb6-9901-f12d9155ec34 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:23.669013 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c867b3-8eaf-4bb6-9901-f12d9155ec34 admin admin] PATH_INFO: `/v3/users/bd7803393aa045cb9fe3f20c02fe9a88` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:23.670409 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c867b3-8eaf-4bb6-9901-f12d9155ec34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd7803393aa045cb9fe3f20c02fe9a88)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:23.673372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c867b3-8eaf-4bb6-9901-f12d9155ec34 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:23.685378 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c30d8c-ad26-4a68-88f5-5e00f52533db admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:23.691295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4b102-913e-4ae1-925a-71bd2f71fb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:23.692364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4b102-913e-4ae1-925a-71bd2f71fb27 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:23.692763 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4b102-913e-4ae1-925a-71bd2f71fb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:23.693134 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4b102-913e-4ae1-925a-71bd2f71fb27 admin admin] PATH_INFO: `/v3/projects/c8949ff1908e49479118955b1ae7dc5e/users/15bd79fc327e4d7bb0bac435fc2550e1/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:23.717292 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4b102-913e-4ae1-925a-71bd2f71fb27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8949ff1908e49479118955b1ae7dc5e, user_id=15bd79fc327e4d7bb0bac435fc2550e1, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:23.721147 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4b102-913e-4ae1-925a-71bd2f71fb27 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:23.733761 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d6c867b3-8eaf-4bb6-9901-f12d9155ec34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd7803393aa045cb9fe3f20c02fe9a88'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:23.737263 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 853/1697] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:23 2022] PUT /identity/v3/projects/c8949ff1908e49479118955b1ae7dc5e/users/15bd79fc327e4d7bb0bac435fc2550e1/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:23.738396 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4b102-913e-4ae1-925a-71bd2f71fb27 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:23.744442 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d6c867b3-8eaf-4bb6-9901-f12d9155ec34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd7803393aa045cb9fe3f20c02fe9a88'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:23.745015 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d6c867b3-8eaf-4bb6-9901-f12d9155ec34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd7803393aa045cb9fe3f20c02fe9a88'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:23.748607 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9fc4b102-913e-4ae1-925a-71bd2f71fb27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:23.767449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d6c867b3-8eaf-4bb6-9901-f12d9155ec34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd7803393aa045cb9fe3f20c02fe9a88'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:23.768194 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d6c867b3-8eaf-4bb6-9901-f12d9155ec34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd7803393aa045cb9fe3f20c02fe9a88'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:23.800958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 845/1698] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:23 2022] DELETE /identity/v3/users/bd7803393aa045cb9fe3f20c02fe9a88 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:23.802292 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ac3d2-c9ab-4e5c-a7ff-1ffcaa3d94f7 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:23.802633 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ac3d2-c9ab-4e5c-a7ff-1ffcaa3d94f7 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:23.803037 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ac3d2-c9ab-4e5c-a7ff-1ffcaa3d94f7 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:23.806582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 846/1699] 10.222.0.23 () {54 vars in 961 bytes} [Mon May 9 23:02:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 23:02:23.807735 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac789b31-0f3a-461e-82eb-9abb4ab60593 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:23.808020 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac789b31-0f3a-461e-82eb-9abb4ab60593 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:23.808294 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac789b31-0f3a-461e-82eb-9abb4ab60593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:23.843906 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ac789b31-0f3a-461e-82eb-9abb4ab60593 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:23.844892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ac789b31-0f3a-461e-82eb-9abb4ab60593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:23.895172 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4b102-913e-4ae1-925a-71bd2f71fb27 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:23.995736 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 847/1700] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:23.997607 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574d10c-5c4d-49be-9b7a-db9a30d2b93f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:23.997994 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574d10c-5c4d-49be-9b7a-db9a30d2b93f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:23.998564 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574d10c-5c4d-49be-9b7a-db9a30d2b93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:24.048721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969523fd-ba96-42dc-ac02-da312c94091e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf9bc2fa959d4706b9f9e87509829059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:24.048721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969523fd-ba96-42dc-ac02-da312c94091e service placement] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:24.048965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969523fd-ba96-42dc-ac02-da312c94091e service placement] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:24.049239 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969523fd-ba96-42dc-ac02-da312c94091e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:24.055776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-969523fd-ba96-42dc-ac02-da312c94091e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:24.063689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-7574d10c-5c4d-49be-9b7a-db9a30d2b93f None None] MFA Rules not processed for user `15bd79fc327e4d7bb0bac435fc2550e1`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:24.064924 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7574d10c-5c4d-49be-9b7a-db9a30d2b93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:24.081141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969523fd-ba96-42dc-ac02-da312c94091e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:24.084264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969523fd-ba96-42dc-ac02-da312c94091e service placement] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:24.109689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 854/1701] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:24.112860 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fac85-aa1f-4e0e-be5c-a543d462a33c None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:24.116416 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fac85-aa1f-4e0e-be5c-a543d462a33c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:24.118059 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fac85-aa1f-4e0e-be5c-a543d462a33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:24.161848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c74fac85-aa1f-4e0e-be5c-a543d462a33c None None] MFA Rules not processed for user `02fcd7cc86134ce488c7dd75dfb174b8`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:24.165426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c74fac85-aa1f-4e0e-be5c-a543d462a33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:24.243325 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 848/1702] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:24.249255 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14279030-3184-4f79-9389-0b3be0aa4a5c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:24.252038 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14279030-3184-4f79-9389-0b3be0aa4a5c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:24.252793 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14279030-3184-4f79-9389-0b3be0aa4a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:24.317909 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-14279030-3184-4f79-9389-0b3be0aa4a5c None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:24.329672 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-14279030-3184-4f79-9389-0b3be0aa4a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:24.356644 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 849/1703] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:24.358048 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14279030-3184-4f79-9389-0b3be0aa4a5c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:24.363188 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-14279030-3184-4f79-9389-0b3be0aa4a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:24.385100 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 855/1704] 10.222.0.23 () {58 vars in 1058 bytes} [Mon May 9 23:02:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:24.387708 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fac85-aa1f-4e0e-be5c-a543d462a33c None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:24.393599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c74fac85-aa1f-4e0e-be5c-a543d462a33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:24.399505 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14279030-3184-4f79-9389-0b3be0aa4a5c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:24.418363 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31cbef-8493-48c0-a494-f980605e5c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:24.419268 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31cbef-8493-48c0-a494-f980605e5c13 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:24.419679 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31cbef-8493-48c0-a494-f980605e5c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:24.420279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31cbef-8493-48c0-a494-f980605e5c13 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:24.422076 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31cbef-8493-48c0-a494-f980605e5c13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:24.429400 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31cbef-8493-48c0-a494-f980605e5c13 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:24.431102 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fac85-aa1f-4e0e-be5c-a543d462a33c None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:24.445101 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea46ae-ca4e-4b8d-80a0-14207abf6680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:24.446149 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea46ae-ca4e-4b8d-80a0-14207abf6680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:24.446529 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea46ae-ca4e-4b8d-80a0-14207abf6680 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:24.446912 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea46ae-ca4e-4b8d-80a0-14207abf6680 admin admin] PATH_INFO: `/v3/projects/bb01e86c8dbe4e5b986653458c907ef8` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:24.449852 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea46ae-ca4e-4b8d-80a0-14207abf6680 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb01e86c8dbe4e5b986653458c907ef8)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:24.451969 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea46ae-ca4e-4b8d-80a0-14207abf6680 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:24.493006 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 850/1705] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:24 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:24.495175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0942c-5288-43b4-9a3e-6d9e3828fa58 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:24.496300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0942c-5288-43b4-9a3e-6d9e3828fa58 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:24.496973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0942c-5288-43b4-9a3e-6d9e3828fa58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:24.562774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-86f0942c-5288-43b4-9a3e-6d9e3828fa58 None None] MFA Rules not processed for user `84a45506c19c4cda8173c164a097f925`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:24.564460 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-86f0942c-5288-43b4-9a3e-6d9e3828fa58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:24.565231 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-01ea46ae-ca4e-4b8d-80a0-14207abf6680 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb01e86c8dbe4e5b986653458c907ef8', 'request_id': 'req-01ea46ae-ca4e-4b8d-80a0-14207abf6680'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:24.579033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-01ea46ae-ca4e-4b8d-80a0-14207abf6680 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb01e86c8dbe4e5b986653458c907ef8', 'request_id': 'req-01ea46ae-ca4e-4b8d-80a0-14207abf6680'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:24.594150 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-01ea46ae-ca4e-4b8d-80a0-14207abf6680 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb01e86c8dbe4e5b986653458c907ef8', 'request_id': 'req-01ea46ae-ca4e-4b8d-80a0-14207abf6680'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:24.637029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-01ea46ae-ca4e-4b8d-80a0-14207abf6680 admin admin] The token cache is being invalidate because project bb01e86c8dbe4e5b986653458c907ef8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:24.637685 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-01ea46ae-ca4e-4b8d-80a0-14207abf6680 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:24.639922 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-01ea46ae-ca4e-4b8d-80a0-14207abf6680 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:24.647384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 856/1706] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:24 2022] DELETE /identity/v3/projects/bb01e86c8dbe4e5b986653458c907ef8 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:24.647384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea46ae-ca4e-4b8d-80a0-14207abf6680 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:24.649388 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-01ea46ae-ca4e-4b8d-80a0-14207abf6680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:24.755079 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 851/1707] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:24.758403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b32168-74d5-4b42-a7df-fea013e84619 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:24.758819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b32168-74d5-4b42-a7df-fea013e84619 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:24.759800 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b32168-74d5-4b42-a7df-fea013e84619 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:24.764834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 852/1708] 10.222.0.23 () {54 vars in 912 bytes} [Mon May 9 23:02:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 23:02:24.770074 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b32168-74d5-4b42-a7df-fea013e84619 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:24.774255 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c9b32168-74d5-4b42-a7df-fea013e84619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:24.828549 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea46ae-ca4e-4b8d-80a0-14207abf6680 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:24.845854 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6b0b5-2d58-48cf-8393-ccc56a261dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:24.846766 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6b0b5-2d58-48cf-8393-ccc56a261dbf admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:24.847161 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6b0b5-2d58-48cf-8393-ccc56a261dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:24.849256 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6b0b5-2d58-48cf-8393-ccc56a261dbf admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:24.849863 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d6b0b5-2d58-48cf-8393-ccc56a261dbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:02:24.850295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d6b0b5-2d58-48cf-8393-ccc56a261dbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:24.858422 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d6b0b5-2d58-48cf-8393-ccc56a261dbf admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:24.884255 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 857/1709] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:02:24.887460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8dc5f-20c5-4c82-a42e-bf58b356126a None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:24.890246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8dc5f-20c5-4c82-a42e-bf58b356126a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:24.890666 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8dc5f-20c5-4c82-a42e-bf58b356126a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:24.926911 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b32168-74d5-4b42-a7df-fea013e84619 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:24.933874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f0c8dc5f-20c5-4c82-a42e-bf58b356126a None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:24.935407 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f0c8dc5f-20c5-4c82-a42e-bf58b356126a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:24.940537 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ac3f8-e0c7-450d-b8ae-b0aa50154db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:24.941390 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ac3f8-e0c7-450d-b8ae-b0aa50154db0 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:24.941779 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ac3f8-e0c7-450d-b8ae-b0aa50154db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:24.942372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ac3f8-e0c7-450d-b8ae-b0aa50154db0 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:24.943216 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ac3f8-e0c7-450d-b8ae-b0aa50154db0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:24.947337 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ac3f8-e0c7-450d-b8ae-b0aa50154db0 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:24.962314 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 858/1710] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:24.963328 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8dc5f-20c5-4c82-a42e-bf58b356126a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:24.966930 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f0c8dc5f-20c5-4c82-a42e-bf58b356126a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:25.009875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 853/1711] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:24 2022] POST /identity/v3/users => generated 299 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:25.011213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469e662-69a2-47c1-8df1-a553526fe161 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:25.012704 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469e662-69a2-47c1-8df1-a553526fe161 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:25.013102 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469e662-69a2-47c1-8df1-a553526fe161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:25.052090 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-c469e662-69a2-47c1-8df1-a553526fe161 None None] MFA Rules not processed for user `02fcd7cc86134ce488c7dd75dfb174b8`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:25.055127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c469e662-69a2-47c1-8df1-a553526fe161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:25.166993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8dc5f-20c5-4c82-a42e-bf58b356126a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:25.182378 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18a4b3-d7ce-4f5e-8eb7-474238146f00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:25.183233 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18a4b3-d7ce-4f5e-8eb7-474238146f00 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:25.184778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18a4b3-d7ce-4f5e-8eb7-474238146f00 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:25.186361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18a4b3-d7ce-4f5e-8eb7-474238146f00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:25.207398 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18a4b3-d7ce-4f5e-8eb7-474238146f00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:25.210642 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18a4b3-d7ce-4f5e-8eb7-474238146f00 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:25.233451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 854/1712] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 9 23:02:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:25.234604 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c469e662-69a2-47c1-8df1-a553526fe161 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:25.236439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 859/1713] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:25.237953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca01239-44fb-4f5c-8abb-95d3f08be9fb None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:25.241080 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca01239-44fb-4f5c-8abb-95d3f08be9fb None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:25.241474 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca01239-44fb-4f5c-8abb-95d3f08be9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:25.255248 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c469e662-69a2-47c1-8df1-a553526fe161 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:25.271778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd35438-5c17-4ff2-916e-9b4f26755e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:25.271778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd35438-5c17-4ff2-916e-9b4f26755e7c admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:25.271778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd35438-5c17-4ff2-916e-9b4f26755e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:25.272349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd35438-5c17-4ff2-916e-9b4f26755e7c admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:25.273157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd35438-5c17-4ff2-916e-9b4f26755e7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:25.277297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd35438-5c17-4ff2-916e-9b4f26755e7c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:25.279645 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-2ca01239-44fb-4f5c-8abb-95d3f08be9fb None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:25.281458 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-2ca01239-44fb-4f5c-8abb-95d3f08be9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:25.307211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 860/1714] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:25.307211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca01239-44fb-4f5c-8abb-95d3f08be9fb None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:25.333041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 855/1715] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:25 2022] POST /identity/v3/projects => generated 369 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:25.333806 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca01239-44fb-4f5c-8abb-95d3f08be9fb None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:25.337773 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd35438-5c17-4ff2-916e-9b4f26755e7c admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:25.476342 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0a781-a9f7-4cf9-b65c-598dc0db54a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:25.477488 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0a781-a9f7-4cf9-b65c-598dc0db54a0 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:25.477764 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0a781-a9f7-4cf9-b65c-598dc0db54a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:25.478175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0a781-a9f7-4cf9-b65c-598dc0db54a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:25.478820 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0a781-a9f7-4cf9-b65c-598dc0db54a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:25.487828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0a781-a9f7-4cf9-b65c-598dc0db54a0 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:25.534228 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 861/1716] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:25.534228 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0a781-a9f7-4cf9-b65c-598dc0db54a0 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:25.575384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0a781-a9f7-4cf9-b65c-598dc0db54a0 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:25.596363 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002c4a7-a4aa-4872-ac71-680382199b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:25.597057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002c4a7-a4aa-4872-ac71-680382199b87 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:25.597719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002c4a7-a4aa-4872-ac71-680382199b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:25.597719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002c4a7-a4aa-4872-ac71-680382199b87 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:25.598318 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c002c4a7-a4aa-4872-ac71-680382199b87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:25.600248 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c002c4a7-a4aa-4872-ac71-680382199b87 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:25.613693 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd35438-5c17-4ff2-916e-9b4f26755e7c admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:25.704501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7c167-4088-43ba-b4ec-29f63bb77ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:25.705389 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7c167-4088-43ba-b4ec-29f63bb77ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:25.705814 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7c167-4088-43ba-b4ec-29f63bb77ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:25.706409 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7c167-4088-43ba-b4ec-29f63bb77ec2 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:25.707354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a7c167-4088-43ba-b4ec-29f63bb77ec2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:02:25.710776 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a7c167-4088-43ba-b4ec-29f63bb77ec2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:25.723581 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a7c167-4088-43ba-b4ec-29f63bb77ec2 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:25.753891 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 862/1717] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:25 2022] POST /identity/v3/users => generated 285 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:25.755277 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002c4a7-a4aa-4872-ac71-680382199b87 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:25.765465 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 856/1718] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:02:25.766686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c002c4a7-a4aa-4872-ac71-680382199b87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:25.779782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7c167-4088-43ba-b4ec-29f63bb77ec2 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:25.801291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7c167-4088-43ba-b4ec-29f63bb77ec2 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:25.822564 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ea7fe-69af-4ac3-9437-715e91ed265c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:25.827407 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ea7fe-69af-4ac3-9437-715e91ed265c admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:25.827835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ea7fe-69af-4ac3-9437-715e91ed265c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:25.828219 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ea7fe-69af-4ac3-9437-715e91ed265c admin admin] PATH_INFO: `/v3/projects/a266a38f49b049e1bd5e233b55fad7c6/users/f5a79944a2964c67b3e1eea4d67b6fff/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:25.848573 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ea7fe-69af-4ac3-9437-715e91ed265c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a266a38f49b049e1bd5e233b55fad7c6, user_id=f5a79944a2964c67b3e1eea4d67b6fff, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:25.851483 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ea7fe-69af-4ac3-9437-715e91ed265c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:25.879431 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 857/1719] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:25 2022] PUT /identity/v3/projects/a266a38f49b049e1bd5e233b55fad7c6/users/f5a79944a2964c67b3e1eea4d67b6fff/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:25.880237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ea7fe-69af-4ac3-9437-715e91ed265c admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:25.941959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002c4a7-a4aa-4872-ac71-680382199b87 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:26.049125 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3ffa1-2e0c-4407-9651-2ad03ec5698c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:26.049798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3ffa1-2e0c-4407-9651-2ad03ec5698c service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:26.050081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3ffa1-2e0c-4407-9651-2ad03ec5698c service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:26.050381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3ffa1-2e0c-4407-9651-2ad03ec5698c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:26.092870 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ea7fe-69af-4ac3-9437-715e91ed265c admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:26.103503 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae8344-da01-453e-bf28-5509dbdf28c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:26.105766 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae8344-da01-453e-bf28-5509dbdf28c6 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:26.106055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae8344-da01-453e-bf28-5509dbdf28c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:26.106367 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae8344-da01-453e-bf28-5509dbdf28c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:26.109397 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae8344-da01-453e-bf28-5509dbdf28c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:26.118890 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae8344-da01-453e-bf28-5509dbdf28c6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:26.150589 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 858/1720] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:26.151993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae8344-da01-453e-bf28-5509dbdf28c6 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:26.190285 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae8344-da01-453e-bf28-5509dbdf28c6 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:26.191478 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3ffa1-2e0c-4407-9651-2ad03ec5698c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:26.192690 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3ffa1-2e0c-4407-9651-2ad03ec5698c service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:26.204872 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e4ea8-ef9c-4587-b0d7-0efa98877d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:26.205600 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e4ea8-ef9c-4587-b0d7-0efa98877d0f admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:26.206174 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e4ea8-ef9c-4587-b0d7-0efa98877d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:26.206790 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e4ea8-ef9c-4587-b0d7-0efa98877d0f admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:26.207737 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407e4ea8-ef9c-4587-b0d7-0efa98877d0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:26.212792 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407e4ea8-ef9c-4587-b0d7-0efa98877d0f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:26.236147 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 863/1721] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:26.237018 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3ffa1-2e0c-4407-9651-2ad03ec5698c service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:26.274974 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3ffa1-2e0c-4407-9651-2ad03ec5698c service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:26.294827 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f06f6e-0e0f-4176-9dd2-1a388fba2be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:26.295578 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f06f6e-0e0f-4176-9dd2-1a388fba2be8 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:26.295806 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f06f6e-0e0f-4176-9dd2-1a388fba2be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:26.296084 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f06f6e-0e0f-4176-9dd2-1a388fba2be8 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:26.296820 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f06f6e-0e0f-4176-9dd2-1a388fba2be8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:26.298976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f06f6e-0e0f-4176-9dd2-1a388fba2be8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:26.312710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 859/1722] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:26 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:26.315245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e4ea8-ef9c-4587-b0d7-0efa98877d0f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:26.324521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 864/1723] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:26.325475 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f06f6e-0e0f-4176-9dd2-1a388fba2be8 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:26.327420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-73f06f6e-0e0f-4176-9dd2-1a388fba2be8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:26.348347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f06f6e-0e0f-4176-9dd2-1a388fba2be8 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:26.485667 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1261286-fe9d-414d-a8b0-9a447729a76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:26.486575 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1261286-fe9d-414d-a8b0-9a447729a76a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:26.486857 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1261286-fe9d-414d-a8b0-9a447729a76a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:26.491875 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1261286-fe9d-414d-a8b0-9a447729a76a admin admin] PATH_INFO: `/v3/users/3dc4b31fcb154251ac27e4e4dce5e6c7` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:26.496433 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1261286-fe9d-414d-a8b0-9a447729a76a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dc4b31fcb154251ac27e4e4dce5e6c7)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:26.499262 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1261286-fe9d-414d-a8b0-9a447729a76a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:26.513722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e4ea8-ef9c-4587-b0d7-0efa98877d0f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:26.531412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8d610-3deb-4902-9307-29767e27f60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:26.533091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8d610-3deb-4902-9307-29767e27f60e admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:26.534105 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8d610-3deb-4902-9307-29767e27f60e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:26.534900 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8d610-3deb-4902-9307-29767e27f60e admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:26.536414 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e8d610-3deb-4902-9307-29767e27f60e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:26.541453 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e8d610-3deb-4902-9307-29767e27f60e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:26.590628 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 860/1724] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:26.591552 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8d610-3deb-4902-9307-29767e27f60e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:26.623819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8d610-3deb-4902-9307-29767e27f60e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:26.649382 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea137ed-80ac-436c-b78c-fe3830e23509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:26.650103 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea137ed-80ac-436c-b78c-fe3830e23509 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:26.650418 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea137ed-80ac-436c-b78c-fe3830e23509 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:26.650694 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea137ed-80ac-436c-b78c-fe3830e23509 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:26.651451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea137ed-80ac-436c-b78c-fe3830e23509 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:26.653343 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea137ed-80ac-436c-b78c-fe3830e23509 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:26.665104 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e1261286-fe9d-414d-a8b0-9a447729a76a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dc4b31fcb154251ac27e4e4dce5e6c7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:26.683073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e1261286-fe9d-414d-a8b0-9a447729a76a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dc4b31fcb154251ac27e4e4dce5e6c7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:26.712055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e1261286-fe9d-414d-a8b0-9a447729a76a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dc4b31fcb154251ac27e4e4dce5e6c7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:26.713203 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e1261286-fe9d-414d-a8b0-9a447729a76a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dc4b31fcb154251ac27e4e4dce5e6c7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:26.720089 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 861/1725] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:26 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:26.720902 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea137ed-80ac-436c-b78c-fe3830e23509 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:26.737312 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea137ed-80ac-436c-b78c-fe3830e23509 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:26.754040 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e1261286-fe9d-414d-a8b0-9a447729a76a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dc4b31fcb154251ac27e4e4dce5e6c7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:26.754279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2bf58-39b4-482e-b334-f488a834b362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:26.755147 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2bf58-39b4-482e-b334-f488a834b362 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:26.755434 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2bf58-39b4-482e-b334-f488a834b362 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:26.755802 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2bf58-39b4-482e-b334-f488a834b362 admin admin] PATH_INFO: `/v3/projects/a266a38f49b049e1bd5e233b55fad7c6/users/f5a79944a2964c67b3e1eea4d67b6fff/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:26.765586 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc2bf58-39b4-482e-b334-f488a834b362 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a266a38f49b049e1bd5e233b55fad7c6, user_id=f5a79944a2964c67b3e1eea4d67b6fff, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:26.769279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc2bf58-39b4-482e-b334-f488a834b362 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:26.794423 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 865/1726] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:26 2022] DELETE /identity/v3/users/3dc4b31fcb154251ac27e4e4dce5e6c7 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:26.801293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1261286-fe9d-414d-a8b0-9a447729a76a admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:26.807922 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e1261286-fe9d-414d-a8b0-9a447729a76a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:26.820395 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 862/1727] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:26 2022] PUT /identity/v3/projects/a266a38f49b049e1bd5e233b55fad7c6/users/f5a79944a2964c67b3e1eea4d67b6fff/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:26.821415 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2bf58-39b4-482e-b334-f488a834b362 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:26.823502 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8bc2bf58-39b4-482e-b334-f488a834b362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:26.944552 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1261286-fe9d-414d-a8b0-9a447729a76a admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:26.947856 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2bf58-39b4-482e-b334-f488a834b362 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:26.953004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6819f-9d74-4378-98c3-a0a584db0c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:26.954573 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6819f-9d74-4378-98c3-a0a584db0c8d admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:26.954927 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6819f-9d74-4378-98c3-a0a584db0c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:26.955278 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6819f-9d74-4378-98c3-a0a584db0c8d admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:26.956860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6819f-9d74-4378-98c3-a0a584db0c8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:26.959056 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6819f-9d74-4378-98c3-a0a584db0c8d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:26.975900 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 863/1728] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:26.979097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc758b98-933f-4731-941a-c298de2675e6 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:26.979509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc758b98-933f-4731-941a-c298de2675e6 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:26.980305 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc758b98-933f-4731-941a-c298de2675e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:26.980855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63d5be-f044-4057-932a-07a6335a7518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:26.986405 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63d5be-f044-4057-932a-07a6335a7518 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:26.988118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63d5be-f044-4057-932a-07a6335a7518 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:26.989184 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63d5be-f044-4057-932a-07a6335a7518 admin admin] PATH_INFO: `/v3/projects/c19350cdee8a47a7835ff7727d949f60/users/619b722988f74c0fab61b0486efb1a21/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:27.028098 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-bc758b98-933f-4731-941a-c298de2675e6 None None] MFA Rules not processed for user `f5a79944a2964c67b3e1eea4d67b6fff`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:27.030403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-bc758b98-933f-4731-941a-c298de2675e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:27.035861 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa63d5be-f044-4057-932a-07a6335a7518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c19350cdee8a47a7835ff7727d949f60, user_id=619b722988f74c0fab61b0486efb1a21, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:27.041669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa63d5be-f044-4057-932a-07a6335a7518 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:27.084952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 866/1729] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:26 2022] PUT /identity/v3/projects/c19350cdee8a47a7835ff7727d949f60/users/619b722988f74c0fab61b0486efb1a21/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:27.086276 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63d5be-f044-4057-932a-07a6335a7518 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:27.329422 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 864/1730] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:27.330819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeee936-a5fd-49c1-bf5d-44a7a1874932 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:27.333070 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeee936-a5fd-49c1-bf5d-44a7a1874932 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:27.333832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeee936-a5fd-49c1-bf5d-44a7a1874932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:27.370342 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63d5be-f044-4057-932a-07a6335a7518 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:27.382578 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecfe99-22b5-445d-b398-ac70213d7942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:27.383547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecfe99-22b5-445d-b398-ac70213d7942 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:27.383847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecfe99-22b5-445d-b398-ac70213d7942 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:27.384111 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecfe99-22b5-445d-b398-ac70213d7942 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:27.384847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ecfe99-22b5-445d-b398-ac70213d7942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:27.390444 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ecfe99-22b5-445d-b398-ac70213d7942 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:27.433957 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 867/1731] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:27.436119 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecfe99-22b5-445d-b398-ac70213d7942 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:27.441425 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-38ecfe99-22b5-445d-b398-ac70213d7942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:27.455950 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-3eeee936-a5fd-49c1-bf5d-44a7a1874932 None None] MFA Rules not processed for user `619b722988f74c0fab61b0486efb1a21`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:27.476657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3eeee936-a5fd-49c1-bf5d-44a7a1874932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:27.487506 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecfe99-22b5-445d-b398-ac70213d7942 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:27.512444 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364aa8bb-8495-4f36-a5b9-8ecf01a9378a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:27.513328 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364aa8bb-8495-4f36-a5b9-8ecf01a9378a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:27.513619 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364aa8bb-8495-4f36-a5b9-8ecf01a9378a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:27.514545 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364aa8bb-8495-4f36-a5b9-8ecf01a9378a admin admin] PATH_INFO: `/v3/projects/02f99bf5f6404a1fa6c45c21136c3889` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:27.516875 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364aa8bb-8495-4f36-a5b9-8ecf01a9378a admin admin] RBAC: Authorizing `identity:delete_project(project_id=02f99bf5f6404a1fa6c45c21136c3889)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:27.525543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364aa8bb-8495-4f36-a5b9-8ecf01a9378a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:27.655503 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-364aa8bb-8495-4f36-a5b9-8ecf01a9378a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02f99bf5f6404a1fa6c45c21136c3889', 'request_id': 'req-364aa8bb-8495-4f36-a5b9-8ecf01a9378a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:27.676793 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-364aa8bb-8495-4f36-a5b9-8ecf01a9378a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02f99bf5f6404a1fa6c45c21136c3889', 'request_id': 'req-364aa8bb-8495-4f36-a5b9-8ecf01a9378a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:27.710166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-364aa8bb-8495-4f36-a5b9-8ecf01a9378a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02f99bf5f6404a1fa6c45c21136c3889', 'request_id': 'req-364aa8bb-8495-4f36-a5b9-8ecf01a9378a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:27.752230 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-364aa8bb-8495-4f36-a5b9-8ecf01a9378a admin admin] The token cache is being invalidate because project 02f99bf5f6404a1fa6c45c21136c3889 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:27.753856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-364aa8bb-8495-4f36-a5b9-8ecf01a9378a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:27.754870 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-364aa8bb-8495-4f36-a5b9-8ecf01a9378a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:27.759569 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 868/1732] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:27 2022] DELETE /identity/v3/projects/02f99bf5f6404a1fa6c45c21136c3889 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:27.760573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364aa8bb-8495-4f36-a5b9-8ecf01a9378a admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:27.763157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-364aa8bb-8495-4f36-a5b9-8ecf01a9378a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:27.792887 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 865/1733] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:27 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:27.793981 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeee936-a5fd-49c1-bf5d-44a7a1874932 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:27.815744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeee936-a5fd-49c1-bf5d-44a7a1874932 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:27.877191 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364aa8bb-8495-4f36-a5b9-8ecf01a9378a admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:27.882173 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5cc1a-f144-4398-b9bb-065bc94db6a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:27.883168 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5cc1a-f144-4398-b9bb-065bc94db6a6 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:27.883686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5cc1a-f144-4398-b9bb-065bc94db6a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:27.884161 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5cc1a-f144-4398-b9bb-065bc94db6a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:27.886336 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-03d5cc1a-f144-4398-b9bb-065bc94db6a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:27.940296 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66da4d6-54ab-407d-a99e-d5cf8d906d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:27.941122 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66da4d6-54ab-407d-a99e-d5cf8d906d57 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:27.941402 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66da4d6-54ab-407d-a99e-d5cf8d906d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:27.942316 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66da4d6-54ab-407d-a99e-d5cf8d906d57 admin admin] PATH_INFO: `/v3/projects/f0c37f9323b04350a1a12ffd9ff4acce/users/dd979890cd2040ee8779b8a6a7bdcc41/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:27.973550 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66da4d6-54ab-407d-a99e-d5cf8d906d57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0c37f9323b04350a1a12ffd9ff4acce, user_id=dd979890cd2040ee8779b8a6a7bdcc41, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:27.976264 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66da4d6-54ab-407d-a99e-d5cf8d906d57 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:27.981147 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5cc1a-f144-4398-b9bb-065bc94db6a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:27.983504 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5cc1a-f144-4398-b9bb-065bc94db6a6 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:28.006458 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 869/1734] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:27 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:28.007725 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5cc1a-f144-4398-b9bb-065bc94db6a6 service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:28.009960 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-03d5cc1a-f144-4398-b9bb-065bc94db6a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:28.022188 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 866/1735] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:27 2022] PUT /identity/v3/projects/f0c37f9323b04350a1a12ffd9ff4acce/users/dd979890cd2040ee8779b8a6a7bdcc41/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:28.023450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66da4d6-54ab-407d-a99e-d5cf8d906d57 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:28.028033 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e66da4d6-54ab-407d-a99e-d5cf8d906d57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:28.047010 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5cc1a-f144-4398-b9bb-065bc94db6a6 service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:28.184255 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66da4d6-54ab-407d-a99e-d5cf8d906d57 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:28.196937 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407489a4-ce5b-4374-bfdc-81bc9ae72b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:28.197639 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407489a4-ce5b-4374-bfdc-81bc9ae72b6b admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:28.197889 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407489a4-ce5b-4374-bfdc-81bc9ae72b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:28.198192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407489a4-ce5b-4374-bfdc-81bc9ae72b6b admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:28.198880 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407489a4-ce5b-4374-bfdc-81bc9ae72b6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:28.202155 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407489a4-ce5b-4374-bfdc-81bc9ae72b6b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:28.250270 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf4f38-02a9-4835-badd-7f0b532ee404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:28.252485 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf4f38-02a9-4835-badd-7f0b532ee404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:28.253446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf4f38-02a9-4835-badd-7f0b532ee404 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:28.258849 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf4f38-02a9-4835-badd-7f0b532ee404 admin admin] PATH_INFO: `/v3/users/f4b7863576de4dafa7695defe1e8ed64` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:28.262166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf4f38-02a9-4835-badd-7f0b532ee404 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4b7863576de4dafa7695defe1e8ed64)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:28.280381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf4f38-02a9-4835-badd-7f0b532ee404 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:28.330864 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 867/1736] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:28 2022] POST /identity/v3/projects => generated 367 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:28.336565 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbecf0a6-2723-4ee2-8603-2698218f1f74 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:28.337805 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbecf0a6-2723-4ee2-8603-2698218f1f74 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:28.339941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbecf0a6-2723-4ee2-8603-2698218f1f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:28.404646 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-bbecf0a6-2723-4ee2-8603-2698218f1f74 None None] MFA Rules not processed for user `dd979890cd2040ee8779b8a6a7bdcc41`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:28.405864 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-bbecf0a6-2723-4ee2-8603-2698218f1f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:28.428826 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6aaf4f38-02a9-4835-badd-7f0b532ee404 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4b7863576de4dafa7695defe1e8ed64'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:28.439170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6aaf4f38-02a9-4835-badd-7f0b532ee404 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4b7863576de4dafa7695defe1e8ed64'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:28.463908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6aaf4f38-02a9-4835-badd-7f0b532ee404 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4b7863576de4dafa7695defe1e8ed64'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:28.464868 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6aaf4f38-02a9-4835-badd-7f0b532ee404 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4b7863576de4dafa7695defe1e8ed64'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:28.511981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6aaf4f38-02a9-4835-badd-7f0b532ee404 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4b7863576de4dafa7695defe1e8ed64'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:28.523485 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 870/1737] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:28 2022] DELETE /identity/v3/users/f4b7863576de4dafa7695defe1e8ed64 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:28.528907 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf4f38-02a9-4835-badd-7f0b532ee404 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:28.533763 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6aaf4f38-02a9-4835-badd-7f0b532ee404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:28.563188 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 868/1738] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:28.563188 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbecf0a6-2723-4ee2-8603-2698218f1f74 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:28.563188 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-bbecf0a6-2723-4ee2-8603-2698218f1f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:28.748959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf4f38-02a9-4835-badd-7f0b532ee404 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:28.755109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead9b729-34c6-46f1-8ada-14c62b777fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:28.756049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9b729-34c6-46f1-8ada-14c62b777fc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:28.756332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9b729-34c6-46f1-8ada-14c62b777fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:28.756603 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9b729-34c6-46f1-8ada-14c62b777fc8 admin admin] PATH_INFO: `/v3/users/15bd79fc327e4d7bb0bac435fc2550e1` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:28.758518 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead9b729-34c6-46f1-8ada-14c62b777fc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15bd79fc327e4d7bb0bac435fc2550e1)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:28.760453 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead9b729-34c6-46f1-8ada-14c62b777fc8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:28.762011 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbecf0a6-2723-4ee2-8603-2698218f1f74 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:28.782513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21975621-8035-4dda-bd30-78d44d74e637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:28.783456 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21975621-8035-4dda-bd30-78d44d74e637 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:28.783914 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21975621-8035-4dda-bd30-78d44d74e637 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:28.784360 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21975621-8035-4dda-bd30-78d44d74e637 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:28.785238 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21975621-8035-4dda-bd30-78d44d74e637 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:28.789284 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21975621-8035-4dda-bd30-78d44d74e637 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:28.828743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 869/1739] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:28 2022] POST /identity/v3/users => generated 290 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:28.830302 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae5081-ea46-49bf-99f0-adcae1ee3d8f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:28.830797 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae5081-ea46-49bf-99f0-adcae1ee3d8f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:28.831084 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae5081-ea46-49bf-99f0-adcae1ee3d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:28.837201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ead9b729-34c6-46f1-8ada-14c62b777fc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15bd79fc327e4d7bb0bac435fc2550e1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:28.847413 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ead9b729-34c6-46f1-8ada-14c62b777fc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15bd79fc327e4d7bb0bac435fc2550e1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:28.862683 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ead9b729-34c6-46f1-8ada-14c62b777fc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15bd79fc327e4d7bb0bac435fc2550e1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:28.863310 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ead9b729-34c6-46f1-8ada-14c62b777fc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15bd79fc327e4d7bb0bac435fc2550e1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:28.866657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-fbae5081-ea46-49bf-99f0-adcae1ee3d8f None None] MFA Rules not processed for user `1a6f50dcebf146baae8a9f0b5b112795`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:28.868123 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-fbae5081-ea46-49bf-99f0-adcae1ee3d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:28.893619 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ead9b729-34c6-46f1-8ada-14c62b777fc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15bd79fc327e4d7bb0bac435fc2550e1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:28.916111 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 871/1740] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:28 2022] DELETE /identity/v3/users/15bd79fc327e4d7bb0bac435fc2550e1 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:28.917373 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead9b729-34c6-46f1-8ada-14c62b777fc8 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:28.919173 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ead9b729-34c6-46f1-8ada-14c62b777fc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:29.002278 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 870/1741] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 9 23:02:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:29.003990 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46fd60-ba66-4158-a8d1-69007f457f5a None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:29.004444 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46fd60-ba66-4158-a8d1-69007f457f5a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:29.004888 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46fd60-ba66-4158-a8d1-69007f457f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:29.072815 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead9b729-34c6-46f1-8ada-14c62b777fc8 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:29.092058 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1278251-dcf8-4d9e-90d0-d0b33c4b766f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:29.092782 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1278251-dcf8-4d9e-90d0-d0b33c4b766f admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:29.093055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1278251-dcf8-4d9e-90d0-d0b33c4b766f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:29.093326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1278251-dcf8-4d9e-90d0-d0b33c4b766f admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:29.094027 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1278251-dcf8-4d9e-90d0-d0b33c4b766f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:29.096893 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1278251-dcf8-4d9e-90d0-d0b33c4b766f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:29.100258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-af46fd60-ba66-4158-a8d1-69007f457f5a None None] MFA Rules not processed for user `e41b4feaa9be44e89210b70c19ef10ab`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:29.101505 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-af46fd60-ba66-4158-a8d1-69007f457f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:29.163026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 872/1742] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:28 2022] POST /identity/v3/projects => generated 385 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:29.163026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1278251-dcf8-4d9e-90d0-d0b33c4b766f admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:29.163026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c1278251-dcf8-4d9e-90d0-d0b33c4b766f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:29.284699 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 871/1743] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:29.286037 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37d8df-9a1c-4577-ac04-68ab58ac717b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:29.286413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37d8df-9a1c-4577-ac04-68ab58ac717b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:29.286699 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37d8df-9a1c-4577-ac04-68ab58ac717b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:29.316033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1278251-dcf8-4d9e-90d0-d0b33c4b766f admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:29.322800 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0fd35-f3cc-4887-b7ef-07a489b96d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:29.322800 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0fd35-f3cc-4887-b7ef-07a489b96d48 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:29.322800 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0fd35-f3cc-4887-b7ef-07a489b96d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:29.322800 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0fd35-f3cc-4887-b7ef-07a489b96d48 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:29.323790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0fd35-f3cc-4887-b7ef-07a489b96d48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:29.324895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0fd35-f3cc-4887-b7ef-07a489b96d48 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:29.328222 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-3a37d8df-9a1c-4577-ac04-68ab58ac717b None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:29.330019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3a37d8df-9a1c-4577-ac04-68ab58ac717b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:29.350741 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 872/1744] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:29.351781 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37d8df-9a1c-4577-ac04-68ab58ac717b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:29.355270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3a37d8df-9a1c-4577-ac04-68ab58ac717b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:29.363020 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 873/1745] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:29.363020 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0fd35-f3cc-4887-b7ef-07a489b96d48 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:29.393770 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37d8df-9a1c-4577-ac04-68ab58ac717b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:29.403219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0fd35-f3cc-4887-b7ef-07a489b96d48 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:29.407230 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f00499-c767-4da5-b5f4-903b2624b7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:29.408126 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f00499-c767-4da5-b5f4-903b2624b7ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:29.408401 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f00499-c767-4da5-b5f4-903b2624b7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:29.408673 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f00499-c767-4da5-b5f4-903b2624b7ef admin admin] PATH_INFO: `/v3/projects/66d64afef3c14aa19d106c06c7a1f2e2` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:29.409991 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f00499-c767-4da5-b5f4-903b2624b7ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=66d64afef3c14aa19d106c06c7a1f2e2)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:29.412011 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f00499-c767-4da5-b5f4-903b2624b7ef admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:29.417039 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108f562-2d2f-4573-aa22-b8274a03c87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:29.417725 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108f562-2d2f-4573-aa22-b8274a03c87c admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:29.417998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108f562-2d2f-4573-aa22-b8274a03c87c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:29.418295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108f562-2d2f-4573-aa22-b8274a03c87c admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:29.418982 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108f562-2d2f-4573-aa22-b8274a03c87c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:29.420901 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108f562-2d2f-4573-aa22-b8274a03c87c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:29.474647 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 874/1746] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:29 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:29.474647 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108f562-2d2f-4573-aa22-b8274a03c87c admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:29.479634 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b108f562-2d2f-4573-aa22-b8274a03c87c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:29.511526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-77f00499-c767-4da5-b5f4-903b2624b7ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66d64afef3c14aa19d106c06c7a1f2e2', 'request_id': 'req-77f00499-c767-4da5-b5f4-903b2624b7ef'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:29.525604 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-77f00499-c767-4da5-b5f4-903b2624b7ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66d64afef3c14aa19d106c06c7a1f2e2', 'request_id': 'req-77f00499-c767-4da5-b5f4-903b2624b7ef'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:29.547549 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-77f00499-c767-4da5-b5f4-903b2624b7ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66d64afef3c14aa19d106c06c7a1f2e2', 'request_id': 'req-77f00499-c767-4da5-b5f4-903b2624b7ef'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:29.573369 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-77f00499-c767-4da5-b5f4-903b2624b7ef admin admin] The token cache is being invalidate because project 66d64afef3c14aa19d106c06c7a1f2e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:29.573669 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-77f00499-c767-4da5-b5f4-903b2624b7ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:29.574484 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-77f00499-c767-4da5-b5f4-903b2624b7ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:29.582183 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 873/1747] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:29 2022] DELETE /identity/v3/projects/66d64afef3c14aa19d106c06c7a1f2e2 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:29.582183 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb7eaf-b473-4a35-a3ae-062ecb28ac5c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:29.582183 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb7eaf-b473-4a35-a3ae-062ecb28ac5c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:29.582183 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb7eaf-b473-4a35-a3ae-062ecb28ac5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:29.588896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108f562-2d2f-4573-aa22-b8274a03c87c admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:29.618317 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-4deb7eaf-b473-4a35-a3ae-062ecb28ac5c None None] MFA Rules not processed for user `e41b4feaa9be44e89210b70c19ef10ab`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:29.619557 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-4deb7eaf-b473-4a35-a3ae-062ecb28ac5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:29.698102 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72419c5-c7ab-436c-a7bb-ac7c723dfd04 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cb6c294ce1842b68a6bc73ac381ed6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:29.699483 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72419c5-c7ab-436c-a7bb-ac7c723dfd04 service swift] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:29.699878 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72419c5-c7ab-436c-a7bb-ac7c723dfd04 service swift] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:29.700276 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72419c5-c7ab-436c-a7bb-ac7c723dfd04 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:29.702680 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e72419c5-c7ab-436c-a7bb-ac7c723dfd04 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:29.770043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 874/1748] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:29.771288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb7eaf-b473-4a35-a3ae-062ecb28ac5c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:29.781949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-4deb7eaf-b473-4a35-a3ae-062ecb28ac5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:29.829549 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72419c5-c7ab-436c-a7bb-ac7c723dfd04 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:29.835144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72419c5-c7ab-436c-a7bb-ac7c723dfd04 service swift] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:29.854776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 875/1749] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 9 23:02:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 23:02:29.856669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72419c5-c7ab-436c-a7bb-ac7c723dfd04 service swift] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:29.859272 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e72419c5-c7ab-436c-a7bb-ac7c723dfd04 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:29.989317 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb7eaf-b473-4a35-a3ae-062ecb28ac5c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:30.003410 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35735839-5169-4a52-8b57-2056104b061e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:30.004332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35735839-5169-4a52-8b57-2056104b061e admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:30.004731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35735839-5169-4a52-8b57-2056104b061e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:30.016358 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35735839-5169-4a52-8b57-2056104b061e admin admin] PATH_INFO: `/v3/projects/b4705e673a464496ab056abbae151fcf/users/ba2897016aaf42a1a4c1115c58f4f80a/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:30.100389 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35735839-5169-4a52-8b57-2056104b061e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4705e673a464496ab056abbae151fcf, user_id=ba2897016aaf42a1a4c1115c58f4f80a, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:30.104594 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35735839-5169-4a52-8b57-2056104b061e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:30.128001 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72419c5-c7ab-436c-a7bb-ac7c723dfd04 service swift] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:30.136935 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead2c10-06c0-4d77-9c96-9cfae26bcecc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:30.137953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead2c10-06c0-4d77-9c96-9cfae26bcecc service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:30.138083 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead2c10-06c0-4d77-9c96-9cfae26bcecc service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:30.138336 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead2c10-06c0-4d77-9c96-9cfae26bcecc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:30.140622 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9ead2c10-06c0-4d77-9c96-9cfae26bcecc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:30.165089 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 875/1750] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:29 2022] PUT /identity/v3/projects/b4705e673a464496ab056abbae151fcf/users/ba2897016aaf42a1a4c1115c58f4f80a/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:30.168934 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35735839-5169-4a52-8b57-2056104b061e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:30.170615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead2c10-06c0-4d77-9c96-9cfae26bcecc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:30.172082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead2c10-06c0-4d77-9c96-9cfae26bcecc service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:30.174769 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-35735839-5169-4a52-8b57-2056104b061e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:30.197460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 876/1751] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:30.201568 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69503051-c5fd-4b4e-8667-03c19007e5c6 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:30.201859 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69503051-c5fd-4b4e-8667-03c19007e5c6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:30.202155 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69503051-c5fd-4b4e-8667-03c19007e5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:30.255823 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-69503051-c5fd-4b4e-8667-03c19007e5c6 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:30.258269 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-69503051-c5fd-4b4e-8667-03c19007e5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:30.340702 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35735839-5169-4a52-8b57-2056104b061e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:30.362165 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10a259-bed9-4195-8967-86abf77cf6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:30.362913 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10a259-bed9-4195-8967-86abf77cf6a5 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:30.363149 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10a259-bed9-4195-8967-86abf77cf6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:30.367158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10a259-bed9-4195-8967-86abf77cf6a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:30.368010 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10a259-bed9-4195-8967-86abf77cf6a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:30.369660 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10a259-bed9-4195-8967-86abf77cf6a5 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:30.432403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 876/1752] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:30.433410 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10a259-bed9-4195-8967-86abf77cf6a5 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:30.443176 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9a10a259-bed9-4195-8967-86abf77cf6a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:30.491361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 877/1753] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:30.491361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1a719-90b9-4bf3-a6fc-7f56d86037df None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:30.491361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1a719-90b9-4bf3-a6fc-7f56d86037df None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:30.491361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1a719-90b9-4bf3-a6fc-7f56d86037df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:30.536739 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-cdc1a719-90b9-4bf3-a6fc-7f56d86037df None None] MFA Rules not processed for user `96fd3f6dd9e74ba7a4ad159e43e8ae96`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:30.538683 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-cdc1a719-90b9-4bf3-a6fc-7f56d86037df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:30.619249 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 878/1754] 10.222.0.55 () {58 vars in 1015 bytes} [Mon May 9 23:02:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:30.620461 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1a719-90b9-4bf3-a6fc-7f56d86037df None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:30.655448 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1a719-90b9-4bf3-a6fc-7f56d86037df None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:30.670499 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a8661-1857-4cb2-8726-c0d125af73e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:30.671662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a8661-1857-4cb2-8726-c0d125af73e0 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:30.671662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a8661-1857-4cb2-8726-c0d125af73e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:30.672831 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a8661-1857-4cb2-8726-c0d125af73e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:30.673290 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a8661-1857-4cb2-8726-c0d125af73e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:30.675151 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a8661-1857-4cb2-8726-c0d125af73e0 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:30.685031 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10a259-bed9-4195-8967-86abf77cf6a5 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:30.695582 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 879/1755] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:30.695582 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a8661-1857-4cb2-8726-c0d125af73e0 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:30.699348 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a3381-a95c-40b8-8c69-42d8b6f47f76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:30.700188 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a3381-a95c-40b8-8c69-42d8b6f47f76 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:30.700475 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a3381-a95c-40b8-8c69-42d8b6f47f76 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:30.700794 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a3381-a95c-40b8-8c69-42d8b6f47f76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:30.706085 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-662a3381-a95c-40b8-8c69-42d8b6f47f76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:30.711422 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-bd3a8661-1857-4cb2-8726-c0d125af73e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:30.777470 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a8661-1857-4cb2-8726-c0d125af73e0 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:30.792020 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dffd2a-d354-4d42-b963-b77c1e999160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:30.792990 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dffd2a-d354-4d42-b963-b77c1e999160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:30.793316 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dffd2a-d354-4d42-b963-b77c1e999160 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:30.793654 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dffd2a-d354-4d42-b963-b77c1e999160 admin admin] PATH_INFO: `/v3/users/6be2c71368ee41f383832bb693969e89` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:30.795870 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dffd2a-d354-4d42-b963-b77c1e999160 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6be2c71368ee41f383832bb693969e89)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:30.799494 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dffd2a-d354-4d42-b963-b77c1e999160 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:30.870858 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a3381-a95c-40b8-8c69-42d8b6f47f76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:30.872752 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a3381-a95c-40b8-8c69-42d8b6f47f76 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:30.899421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 877/1756] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:30.915166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d2dffd2a-d354-4d42-b963-b77c1e999160 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6be2c71368ee41f383832bb693969e89'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:30.921747 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a3381-a95c-40b8-8c69-42d8b6f47f76 service nova] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:30.939547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d2dffd2a-d354-4d42-b963-b77c1e999160 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6be2c71368ee41f383832bb693969e89'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:30.950522 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a3381-a95c-40b8-8c69-42d8b6f47f76 service nova] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:30.967000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d2dffd2a-d354-4d42-b963-b77c1e999160 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6be2c71368ee41f383832bb693969e89'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:30.967828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d2dffd2a-d354-4d42-b963-b77c1e999160 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6be2c71368ee41f383832bb693969e89'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:30.968988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66fde6-e266-41fd-880d-2e987d346069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:30.969884 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66fde6-e266-41fd-880d-2e987d346069 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:30.970354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66fde6-e266-41fd-880d-2e987d346069 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:30.970776 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66fde6-e266-41fd-880d-2e987d346069 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:30.971642 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a66fde6-e266-41fd-880d-2e987d346069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:30.973416 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a66fde6-e266-41fd-880d-2e987d346069 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:30.986438 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 878/1757] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:30.988110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a10ff8-bd4e-4ca6-af06-f792a26ae0c1 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:30.988545 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a10ff8-bd4e-4ca6-af06-f792a26ae0c1 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:30.989129 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a10ff8-bd4e-4ca6-af06-f792a26ae0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:30.996921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d2dffd2a-d354-4d42-b963-b77c1e999160 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6be2c71368ee41f383832bb693969e89'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:31.012779 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 880/1758] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:30 2022] DELETE /identity/v3/users/6be2c71368ee41f383832bb693969e89 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:31.014039 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dffd2a-d354-4d42-b963-b77c1e999160 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:31.018325 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d2dffd2a-d354-4d42-b963-b77c1e999160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:31.034335 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e4a10ff8-bd4e-4ca6-af06-f792a26ae0c1 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:31.035570 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e4a10ff8-bd4e-4ca6-af06-f792a26ae0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:31.170199 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dffd2a-d354-4d42-b963-b77c1e999160 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:31.182215 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8994043-b931-47d3-b29d-8e5d451daa97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:31.182889 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8994043-b931-47d3-b29d-8e5d451daa97 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:31.183197 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8994043-b931-47d3-b29d-8e5d451daa97 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:31.186829 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8994043-b931-47d3-b29d-8e5d451daa97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:31.191880 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a8994043-b931-47d3-b29d-8e5d451daa97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:31.221553 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8994043-b931-47d3-b29d-8e5d451daa97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:31.226658 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8994043-b931-47d3-b29d-8e5d451daa97 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:31.244408 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 879/1759] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:31.245806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a10ff8-bd4e-4ca6-af06-f792a26ae0c1 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:31.248610 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e4a10ff8-bd4e-4ca6-af06-f792a26ae0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:31.269191 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 881/1760] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:31.273401 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8994043-b931-47d3-b29d-8e5d451daa97 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:31.276720 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a10ff8-bd4e-4ca6-af06-f792a26ae0c1 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:31.278283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a8994043-b931-47d3-b29d-8e5d451daa97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:31.293128 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892c8a7-b1cc-4e89-bfb7-7414f0087273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:31.295066 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892c8a7-b1cc-4e89-bfb7-7414f0087273 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:31.295580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892c8a7-b1cc-4e89-bfb7-7414f0087273 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:31.295983 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892c8a7-b1cc-4e89-bfb7-7414f0087273 admin admin] PATH_INFO: `/v3/projects/b4705e673a464496ab056abbae151fcf/users/ba2897016aaf42a1a4c1115c58f4f80a/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:31.300496 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b892c8a7-b1cc-4e89-bfb7-7414f0087273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4705e673a464496ab056abbae151fcf, user_id=ba2897016aaf42a1a4c1115c58f4f80a, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:31.308979 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b892c8a7-b1cc-4e89-bfb7-7414f0087273 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:31.318039 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8994043-b931-47d3-b29d-8e5d451daa97 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:31.330245 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125a5b7-102c-4cc8-89ab-c0c59d3dce8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:31.332649 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125a5b7-102c-4cc8-89ab-c0c59d3dce8f admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:31.333563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125a5b7-102c-4cc8-89ab-c0c59d3dce8f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:31.333859 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125a5b7-102c-4cc8-89ab-c0c59d3dce8f admin admin] PATH_INFO: `/v3/projects/7bda058f10df4cf0855073bf75b4e9d4/users/07c94732921e4dbe9ac16a49d2b722bc/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:31.350850 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 880/1761] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:31 2022] PUT /identity/v3/projects/b4705e673a464496ab056abbae151fcf/users/ba2897016aaf42a1a4c1115c58f4f80a/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:31.355026 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f7d7c-4471-457c-ab72-8efad062ccd7 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:31.355418 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f7d7c-4471-457c-ab72-8efad062ccd7 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:31.355760 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f7d7c-4471-457c-ab72-8efad062ccd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:31.372328 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0125a5b7-102c-4cc8-89ab-c0c59d3dce8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bda058f10df4cf0855073bf75b4e9d4, user_id=07c94732921e4dbe9ac16a49d2b722bc, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:31.375953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0125a5b7-102c-4cc8-89ab-c0c59d3dce8f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:31.383868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ea3f7d7c-4471-457c-ab72-8efad062ccd7 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:31.385701 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ea3f7d7c-4471-457c-ab72-8efad062ccd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:31.421706 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 882/1762] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:31 2022] PUT /identity/v3/projects/7bda058f10df4cf0855073bf75b4e9d4/users/07c94732921e4dbe9ac16a49d2b722bc/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:31.421706 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125a5b7-102c-4cc8-89ab-c0c59d3dce8f admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:31.529208 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 881/1763] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:31.530945 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927b40d-2bfa-4c4b-a808-b83092bacefe None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:31.531371 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927b40d-2bfa-4c4b-a808-b83092bacefe None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:31.531769 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927b40d-2bfa-4c4b-a808-b83092bacefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:31.583581 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125a5b7-102c-4cc8-89ab-c0c59d3dce8f admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:31.585807 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-3927b40d-2bfa-4c4b-a808-b83092bacefe None None] MFA Rules not processed for user `ba2897016aaf42a1a4c1115c58f4f80a`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:31.587688 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3927b40d-2bfa-4c4b-a808-b83092bacefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:31.590561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e6f3d-3b3a-4eec-beac-074d8681ad1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:31.591463 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e6f3d-3b3a-4eec-beac-074d8681ad1d admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:31.591742 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e6f3d-3b3a-4eec-beac-074d8681ad1d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:31.592016 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e6f3d-3b3a-4eec-beac-074d8681ad1d admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:31.592796 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085e6f3d-3b3a-4eec-beac-074d8681ad1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:02:31.593102 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085e6f3d-3b3a-4eec-beac-074d8681ad1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:31.597024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085e6f3d-3b3a-4eec-beac-074d8681ad1d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:31.629471 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 883/1764] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:02:31.632572 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52011ea9-71e9-4df2-b05f-86610ce7f06f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:31.634598 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52011ea9-71e9-4df2-b05f-86610ce7f06f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:31.634893 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52011ea9-71e9-4df2-b05f-86610ce7f06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:31.712299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-52011ea9-71e9-4df2-b05f-86610ce7f06f None None] MFA Rules not processed for user `07c94732921e4dbe9ac16a49d2b722bc`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:31.723796 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-52011ea9-71e9-4df2-b05f-86610ce7f06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:31.784734 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 882/1765] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:31 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:31.785974 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927b40d-2bfa-4c4b-a808-b83092bacefe None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:31.789855 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3927b40d-2bfa-4c4b-a808-b83092bacefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:31.955038 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 884/1766] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:31.960526 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52011ea9-71e9-4df2-b05f-86610ce7f06f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:31.977357 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52011ea9-71e9-4df2-b05f-86610ce7f06f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:31.986807 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927b40d-2bfa-4c4b-a808-b83092bacefe None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:31.996882 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09156e68-efca-4c7d-ab27-524f909a9559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:31.997644 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09156e68-efca-4c7d-ab27-524f909a9559 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:31.998081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09156e68-efca-4c7d-ab27-524f909a9559 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:31.998398 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09156e68-efca-4c7d-ab27-524f909a9559 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:31.999246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09156e68-efca-4c7d-ab27-524f909a9559 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:32.001701 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09156e68-efca-4c7d-ab27-524f909a9559 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:32.003825 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe39552-955f-4c89-9aa2-87043bde181a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:32.005625 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe39552-955f-4c89-9aa2-87043bde181a service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:32.006398 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe39552-955f-4c89-9aa2-87043bde181a service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:32.006744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe39552-955f-4c89-9aa2-87043bde181a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:32.046482 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe39552-955f-4c89-9aa2-87043bde181a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:32.050762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe39552-955f-4c89-9aa2-87043bde181a service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:32.060010 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 885/1767] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:31 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:32.061029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09156e68-efca-4c7d-ab27-524f909a9559 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:32.063367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-09156e68-efca-4c7d-ab27-524f909a9559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:32.083013 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 883/1768] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:32.088275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61365613-03dd-4be0-96dd-c47a34149b6f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:32.088576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61365613-03dd-4be0-96dd-c47a34149b6f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:32.089492 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61365613-03dd-4be0-96dd-c47a34149b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:32.130948 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-61365613-03dd-4be0-96dd-c47a34149b6f None None] MFA Rules not processed for user `84a45506c19c4cda8173c164a097f925`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:32.132162 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-61365613-03dd-4be0-96dd-c47a34149b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:32.196066 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09156e68-efca-4c7d-ab27-524f909a9559 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:32.208428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630a638-c7ac-4ea8-a40f-b2d6eaed82e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:32.210932 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630a638-c7ac-4ea8-a40f-b2d6eaed82e1 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:32.212783 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630a638-c7ac-4ea8-a40f-b2d6eaed82e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:32.213782 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630a638-c7ac-4ea8-a40f-b2d6eaed82e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:32.338193 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 884/1769] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:32.349957 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61365613-03dd-4be0-96dd-c47a34149b6f None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:32.351691 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-61365613-03dd-4be0-96dd-c47a34149b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:32.369157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630a638-c7ac-4ea8-a40f-b2d6eaed82e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:32.397264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630a638-c7ac-4ea8-a40f-b2d6eaed82e1 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:32.433458 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 886/1770] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:32 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:32.434609 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630a638-c7ac-4ea8-a40f-b2d6eaed82e1 service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:32.618443 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630a638-c7ac-4ea8-a40f-b2d6eaed82e1 service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:32.647398 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f75d9-d14b-4b39-b1f7-d18574879bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:32.651056 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f75d9-d14b-4b39-b1f7-d18574879bdd admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:32.651648 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f75d9-d14b-4b39-b1f7-d18574879bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:32.652081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f75d9-d14b-4b39-b1f7-d18574879bdd admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:32.655768 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f75d9-d14b-4b39-b1f7-d18574879bdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:32.667660 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f75d9-d14b-4b39-b1f7-d18574879bdd admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:32.772643 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 887/1771] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:32 2022] POST /identity/v3/projects => generated 387 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:32.773955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f75d9-d14b-4b39-b1f7-d18574879bdd admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:32.933045 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61365613-03dd-4be0-96dd-c47a34149b6f None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:33.080954 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f75d9-d14b-4b39-b1f7-d18574879bdd admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:33.087399 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33eac0-4515-4a61-9b5a-9fb9cdd09c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:33.088250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33eac0-4515-4a61-9b5a-9fb9cdd09c69 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:33.088647 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33eac0-4515-4a61-9b5a-9fb9cdd09c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:33.089244 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33eac0-4515-4a61-9b5a-9fb9cdd09c69 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:33.090206 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33eac0-4515-4a61-9b5a-9fb9cdd09c69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:33.092606 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33eac0-4515-4a61-9b5a-9fb9cdd09c69 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:33.123510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 888/1772] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:32 2022] POST /identity/v3/users => generated 293 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:33.127569 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d1150-bf77-419a-bd65-41dcfa56f86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:33.128476 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d1150-bf77-419a-bd65-41dcfa56f86f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:33.128753 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d1150-bf77-419a-bd65-41dcfa56f86f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:33.129021 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d1150-bf77-419a-bd65-41dcfa56f86f admin admin] PATH_INFO: `/v3/projects/c8949ff1908e49479118955b1ae7dc5e` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:33.129587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33eac0-4515-4a61-9b5a-9fb9cdd09c69 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:33.140471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d1150-bf77-419a-bd65-41dcfa56f86f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8949ff1908e49479118955b1ae7dc5e)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:33.142454 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d1150-bf77-419a-bd65-41dcfa56f86f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:33.144587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33eac0-4515-4a61-9b5a-9fb9cdd09c69 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:33.155729 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222bb31-c748-4430-b66b-244ee133f6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:33.201694 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222bb31-c748-4430-b66b-244ee133f6a8 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:33.202712 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222bb31-c748-4430-b66b-244ee133f6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:33.203131 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222bb31-c748-4430-b66b-244ee133f6a8 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:33.205110 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b222bb31-c748-4430-b66b-244ee133f6a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:33.207662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b222bb31-c748-4430-b66b-244ee133f6a8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:33.244572 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 889/1773] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:33 2022] POST /identity/v3/users => generated 300 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:33.246313 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222bb31-c748-4430-b66b-244ee133f6a8 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:33.258531 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222bb31-c748-4430-b66b-244ee133f6a8 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:33.319242 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f69d1150-bf77-419a-bd65-41dcfa56f86f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8949ff1908e49479118955b1ae7dc5e', 'request_id': 'req-f69d1150-bf77-419a-bd65-41dcfa56f86f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:33.353698 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f69d1150-bf77-419a-bd65-41dcfa56f86f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8949ff1908e49479118955b1ae7dc5e', 'request_id': 'req-f69d1150-bf77-419a-bd65-41dcfa56f86f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:33.380576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f69d1150-bf77-419a-bd65-41dcfa56f86f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8949ff1908e49479118955b1ae7dc5e', 'request_id': 'req-f69d1150-bf77-419a-bd65-41dcfa56f86f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:33.382696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08410a-17aa-43a4-a541-be4211d81567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:33.386271 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08410a-17aa-43a4-a541-be4211d81567 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:33.386603 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08410a-17aa-43a4-a541-be4211d81567 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:33.386881 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08410a-17aa-43a4-a541-be4211d81567 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:33.387648 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad08410a-17aa-43a4-a541-be4211d81567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:33.393739 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad08410a-17aa-43a4-a541-be4211d81567 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:33.416619 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f69d1150-bf77-419a-bd65-41dcfa56f86f admin admin] The token cache is being invalidate because project c8949ff1908e49479118955b1ae7dc5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:33.418399 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f69d1150-bf77-419a-bd65-41dcfa56f86f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:33.420299 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f69d1150-bf77-419a-bd65-41dcfa56f86f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:33.428870 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 885/1774] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:32 2022] DELETE /identity/v3/projects/c8949ff1908e49479118955b1ae7dc5e => generated 0 bytes in 1082 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:33.430304 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af79f2-af37-409b-a7a9-cfcf147e085c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:33.433069 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af79f2-af37-409b-a7a9-cfcf147e085c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:33.433344 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af79f2-af37-409b-a7a9-cfcf147e085c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:33.437491 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 890/1775] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:33.440063 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08410a-17aa-43a4-a541-be4211d81567 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:33.445907 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ad08410a-17aa-43a4-a541-be4211d81567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:33.481210 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-05af79f2-af37-409b-a7a9-cfcf147e085c None None] MFA Rules not processed for user `e41b4feaa9be44e89210b70c19ef10ab`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:33.483930 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08410a-17aa-43a4-a541-be4211d81567 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:33.490545 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-05af79f2-af37-409b-a7a9-cfcf147e085c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:33.499766 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b69cc1-237f-4a70-9f11-f0ad3d3642e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:33.502229 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b69cc1-237f-4a70-9f11-f0ad3d3642e3 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:33.502569 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b69cc1-237f-4a70-9f11-f0ad3d3642e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:33.502821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b69cc1-237f-4a70-9f11-f0ad3d3642e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:33.512226 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b69cc1-237f-4a70-9f11-f0ad3d3642e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:33.512226 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b69cc1-237f-4a70-9f11-f0ad3d3642e3 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:33.566637 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 891/1776] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:33.583669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b69cc1-237f-4a70-9f11-f0ad3d3642e3 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:33.589077 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e7b69cc1-237f-4a70-9f11-f0ad3d3642e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:33.610970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b69cc1-237f-4a70-9f11-f0ad3d3642e3 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:33.619370 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 886/1777] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:33.620242 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af79f2-af37-409b-a7a9-cfcf147e085c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:33.622494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-05af79f2-af37-409b-a7a9-cfcf147e085c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:33.637966 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af79f2-af37-409b-a7a9-cfcf147e085c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:33.643678 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab4722-22c3-45fc-a0f0-6c506fbd613f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:33.644514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab4722-22c3-45fc-a0f0-6c506fbd613f admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:33.645062 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab4722-22c3-45fc-a0f0-6c506fbd613f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:33.645414 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab4722-22c3-45fc-a0f0-6c506fbd613f admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:33.646227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab4722-22c3-45fc-a0f0-6c506fbd613f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:33.648004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab4722-22c3-45fc-a0f0-6c506fbd613f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:33.654687 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99382615-66df-4387-98de-6b19277a4d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:33.657925 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99382615-66df-4387-98de-6b19277a4d94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:33.659994 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99382615-66df-4387-98de-6b19277a4d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:33.660345 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99382615-66df-4387-98de-6b19277a4d94 admin admin] PATH_INFO: `/v3/users/f5a79944a2964c67b3e1eea4d67b6fff` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:33.662310 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 887/1778] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:33.663504 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab4722-22c3-45fc-a0f0-6c506fbd613f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:33.665995 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99382615-66df-4387-98de-6b19277a4d94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5a79944a2964c67b3e1eea4d67b6fff)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:33.674883 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99382615-66df-4387-98de-6b19277a4d94 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:33.675842 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab4722-22c3-45fc-a0f0-6c506fbd613f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:33.682159 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ed961-ec42-4bcf-a2bb-39e8279f7c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:33.683240 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ed961-ec42-4bcf-a2bb-39e8279f7c66 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:33.683596 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ed961-ec42-4bcf-a2bb-39e8279f7c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:33.683938 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ed961-ec42-4bcf-a2bb-39e8279f7c66 admin admin] PATH_INFO: `/v3/projects/23c33e50fa7e493197afcde22ca11bd0/users/1c91f1634db84bc38de28e16436d330c/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:33.704544 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00ed961-ec42-4bcf-a2bb-39e8279f7c66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23c33e50fa7e493197afcde22ca11bd0, user_id=1c91f1634db84bc38de28e16436d330c, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:33.708294 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00ed961-ec42-4bcf-a2bb-39e8279f7c66 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:33.733066 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 888/1779] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:33 2022] PUT /identity/v3/projects/23c33e50fa7e493197afcde22ca11bd0/users/1c91f1634db84bc38de28e16436d330c/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:33.734175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ed961-ec42-4bcf-a2bb-39e8279f7c66 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:33.861009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-99382615-66df-4387-98de-6b19277a4d94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5a79944a2964c67b3e1eea4d67b6fff'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:33.918397 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-99382615-66df-4387-98de-6b19277a4d94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5a79944a2964c67b3e1eea4d67b6fff'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:33.937974 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ed961-ec42-4bcf-a2bb-39e8279f7c66 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:33.951822 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849d5ca-abe1-4847-b843-7aa893a9386b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:33.953248 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849d5ca-abe1-4847-b843-7aa893a9386b admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:33.954420 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849d5ca-abe1-4847-b843-7aa893a9386b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:33.955406 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849d5ca-abe1-4847-b843-7aa893a9386b admin admin] PATH_INFO: `/v3/projects/f32e8158864e449b9fd607276092ef3f/users/b4eb72b9a3fc43ceb0177148fda513a4/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:33.962182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-99382615-66df-4387-98de-6b19277a4d94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5a79944a2964c67b3e1eea4d67b6fff'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:33.963038 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-99382615-66df-4387-98de-6b19277a4d94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5a79944a2964c67b3e1eea4d67b6fff'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:33.997859 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849d5ca-abe1-4847-b843-7aa893a9386b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f32e8158864e449b9fd607276092ef3f, user_id=b4eb72b9a3fc43ceb0177148fda513a4, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:34.003155 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849d5ca-abe1-4847-b843-7aa893a9386b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:34.009312 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-99382615-66df-4387-98de-6b19277a4d94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5a79944a2964c67b3e1eea4d67b6fff'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:34.033965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 892/1781] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:33 2022] DELETE /identity/v3/users/f5a79944a2964c67b3e1eea4d67b6fff => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:34.033965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99382615-66df-4387-98de-6b19277a4d94 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:34.033965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-99382615-66df-4387-98de-6b19277a4d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:34.035119 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 889/1781] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:33 2022] PUT /identity/v3/projects/f32e8158864e449b9fd607276092ef3f/users/b4eb72b9a3fc43ceb0177148fda513a4/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:34.037284 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09d67b-f422-4035-8e6b-3b5e15681810 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:34.037744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09d67b-f422-4035-8e6b-3b5e15681810 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:34.038347 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09d67b-f422-4035-8e6b-3b5e15681810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:34.139136 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-9c09d67b-f422-4035-8e6b-3b5e15681810 None None] MFA Rules not processed for user `b4eb72b9a3fc43ceb0177148fda513a4`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:34.142875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9c09d67b-f422-4035-8e6b-3b5e15681810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:34.248305 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99382615-66df-4387-98de-6b19277a4d94 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:34.271637 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ed537-29bc-4f25-8f53-b2a6a9dfd7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:34.273210 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ed537-29bc-4f25-8f53-b2a6a9dfd7a7 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:34.277671 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ed537-29bc-4f25-8f53-b2a6a9dfd7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:34.278743 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ed537-29bc-4f25-8f53-b2a6a9dfd7a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:34.283539 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ed537-29bc-4f25-8f53-b2a6a9dfd7a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:34.292661 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ed537-29bc-4f25-8f53-b2a6a9dfd7a7 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:34.364544 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 893/1782] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:34.369307 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ed537-29bc-4f25-8f53-b2a6a9dfd7a7 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:34.377352 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4a0ed537-29bc-4f25-8f53-b2a6a9dfd7a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:34.389517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 890/1783] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:34.390666 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09d67b-f422-4035-8e6b-3b5e15681810 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:34.427711 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ed537-29bc-4f25-8f53-b2a6a9dfd7a7 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:34.429052 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09d67b-f422-4035-8e6b-3b5e15681810 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:34.451577 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc214c-fa44-4d78-b080-6f7291608193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:34.454334 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc214c-fa44-4d78-b080-6f7291608193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:34.455026 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94517351-12ce-4ce2-8e52-f1da56cfc711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:34.456136 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94517351-12ce-4ce2-8e52-f1da56cfc711 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:34.457709 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc214c-fa44-4d78-b080-6f7291608193 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:34.458224 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc214c-fa44-4d78-b080-6f7291608193 admin admin] PATH_INFO: `/v3/projects/a89b506fb2784f79bb7fac2065cc6761` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:34.458728 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94517351-12ce-4ce2-8e52-f1da56cfc711 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:34.461749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94517351-12ce-4ce2-8e52-f1da56cfc711 admin admin] PATH_INFO: `/v3/projects/23c33e50fa7e493197afcde22ca11bd0/users/1c91f1634db84bc38de28e16436d330c/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:34.463820 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc214c-fa44-4d78-b080-6f7291608193 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a89b506fb2784f79bb7fac2065cc6761)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:34.468270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94517351-12ce-4ce2-8e52-f1da56cfc711 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23c33e50fa7e493197afcde22ca11bd0, user_id=1c91f1634db84bc38de28e16436d330c, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:34.471182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc214c-fa44-4d78-b080-6f7291608193 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:34.481002 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94517351-12ce-4ce2-8e52-f1da56cfc711 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:34.529750 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 891/1784] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:34 2022] PUT /identity/v3/projects/23c33e50fa7e493197afcde22ca11bd0/users/1c91f1634db84bc38de28e16436d330c/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:34.543851 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94517351-12ce-4ce2-8e52-f1da56cfc711 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:34.551108 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-94517351-12ce-4ce2-8e52-f1da56cfc711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:34.666378 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-29fc214c-fa44-4d78-b080-6f7291608193 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a89b506fb2784f79bb7fac2065cc6761', 'request_id': 'req-29fc214c-fa44-4d78-b080-6f7291608193'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:34.686487 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-29fc214c-fa44-4d78-b080-6f7291608193 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a89b506fb2784f79bb7fac2065cc6761', 'request_id': 'req-29fc214c-fa44-4d78-b080-6f7291608193'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:34.709967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-29fc214c-fa44-4d78-b080-6f7291608193 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a89b506fb2784f79bb7fac2065cc6761', 'request_id': 'req-29fc214c-fa44-4d78-b080-6f7291608193'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:34.742081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-29fc214c-fa44-4d78-b080-6f7291608193 admin admin] The token cache is being invalidate because project a89b506fb2784f79bb7fac2065cc6761 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:34.742635 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-29fc214c-fa44-4d78-b080-6f7291608193 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:34.743430 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-29fc214c-fa44-4d78-b080-6f7291608193 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:34.745450 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 894/1785] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:34 2022] DELETE /identity/v3/projects/a89b506fb2784f79bb7fac2065cc6761 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:34.747022 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011be0a4-b9fa-4f70-bd7d-f72b208a763b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:34.750890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011be0a4-b9fa-4f70-bd7d-f72b208a763b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:34.751465 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011be0a4-b9fa-4f70-bd7d-f72b208a763b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:34.757008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94517351-12ce-4ce2-8e52-f1da56cfc711 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:34.861358 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-011be0a4-b9fa-4f70-bd7d-f72b208a763b None None] MFA Rules not processed for user `1c91f1634db84bc38de28e16436d330c`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:34.862866 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-011be0a4-b9fa-4f70-bd7d-f72b208a763b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:35.004117 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5de9d-548d-434e-a198-179dcb3e8dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:35.007272 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5de9d-548d-434e-a198-179dcb3e8dbe admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:35.009195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5de9d-548d-434e-a198-179dcb3e8dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:35.009997 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5de9d-548d-434e-a198-179dcb3e8dbe admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:35.013165 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5de9d-548d-434e-a198-179dcb3e8dbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:35.016122 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5de9d-548d-434e-a198-179dcb3e8dbe admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:35.095156 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 892/1786] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:34 2022] POST /identity/v3/projects => generated 385 bytes in 565 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:35.096260 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5de9d-548d-434e-a198-179dcb3e8dbe admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:35.099785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b2d5de9d-548d-434e-a198-179dcb3e8dbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:35.118892 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 895/1787] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:35.118892 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6020f-a37f-42d6-bc4d-ef0367f589aa None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:35.118892 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6020f-a37f-42d6-bc4d-ef0367f589aa None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:35.119195 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6020f-a37f-42d6-bc4d-ef0367f589aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:35.187813 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-42a6020f-a37f-42d6-bc4d-ef0367f589aa None None] MFA Rules not processed for user `96fd3f6dd9e74ba7a4ad159e43e8ae96`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:35.194384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-42a6020f-a37f-42d6-bc4d-ef0367f589aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:35.291865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5de9d-548d-434e-a198-179dcb3e8dbe admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:35.331098 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bf640-9a55-4821-9715-c94efc16374f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:35.333488 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bf640-9a55-4821-9715-c94efc16374f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:35.333759 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bf640-9a55-4821-9715-c94efc16374f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:35.334034 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bf640-9a55-4821-9715-c94efc16374f admin admin] PATH_INFO: `/v3/users/878230f7682348edb7258f1dace88d55` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:35.337573 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6bf640-9a55-4821-9715-c94efc16374f admin admin] RBAC: Authorizing `identity:delete_user(user_id=878230f7682348edb7258f1dace88d55)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:35.339797 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6bf640-9a55-4821-9715-c94efc16374f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:35.380539 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 896/1788] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 9 23:02:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:35.381625 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6020f-a37f-42d6-bc4d-ef0367f589aa None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:35.384623 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-42a6020f-a37f-42d6-bc4d-ef0367f589aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:35.424784 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6020f-a37f-42d6-bc4d-ef0367f589aa None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:35.434982 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0210a-fcbe-4e30-835e-d6eff13f79fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:35.436086 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0210a-fcbe-4e30-835e-d6eff13f79fc admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:35.436439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0210a-fcbe-4e30-835e-d6eff13f79fc admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:35.436786 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0210a-fcbe-4e30-835e-d6eff13f79fc admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:35.442409 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f0210a-fcbe-4e30-835e-d6eff13f79fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:35.451704 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f0210a-fcbe-4e30-835e-d6eff13f79fc admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:35.493375 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 897/1789] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:35 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:35.496310 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0210a-fcbe-4e30-835e-d6eff13f79fc admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:35.505173 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-07f0210a-fcbe-4e30-835e-d6eff13f79fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:35.537112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ef6bf640-9a55-4821-9715-c94efc16374f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '878230f7682348edb7258f1dace88d55'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:35.549201 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ef6bf640-9a55-4821-9715-c94efc16374f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '878230f7682348edb7258f1dace88d55'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:35.549989 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ef6bf640-9a55-4821-9715-c94efc16374f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '878230f7682348edb7258f1dace88d55'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:35.579397 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ef6bf640-9a55-4821-9715-c94efc16374f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '878230f7682348edb7258f1dace88d55'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:35.581715 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ef6bf640-9a55-4821-9715-c94efc16374f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '878230f7682348edb7258f1dace88d55'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:35.620703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0210a-fcbe-4e30-835e-d6eff13f79fc admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:35.626405 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e5075-1a50-4020-b5dc-4b51aa7efe8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:35.628072 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e5075-1a50-4020-b5dc-4b51aa7efe8b service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:35.630039 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e5075-1a50-4020-b5dc-4b51aa7efe8b service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:35.630752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e5075-1a50-4020-b5dc-4b51aa7efe8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:35.633826 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e84e5075-1a50-4020-b5dc-4b51aa7efe8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:35.658518 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 893/1790] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:35 2022] DELETE /identity/v3/users/878230f7682348edb7258f1dace88d55 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:35.659878 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bf640-9a55-4821-9715-c94efc16374f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:35.661876 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ef6bf640-9a55-4821-9715-c94efc16374f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:35.803082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e5075-1a50-4020-b5dc-4b51aa7efe8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:35.806714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e5075-1a50-4020-b5dc-4b51aa7efe8b service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:35.813606 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bf640-9a55-4821-9715-c94efc16374f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:35.840199 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 898/1791] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:35 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:35.841408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e5075-1a50-4020-b5dc-4b51aa7efe8b service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:35.841676 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a11b1c-1413-42bf-8c0b-71d1778c66ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:35.843792 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e84e5075-1a50-4020-b5dc-4b51aa7efe8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:35.847818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a11b1c-1413-42bf-8c0b-71d1778c66ce service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:35.850823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a11b1c-1413-42bf-8c0b-71d1778c66ce service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:35.851113 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a11b1c-1413-42bf-8c0b-71d1778c66ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:35.859069 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-20a11b1c-1413-42bf-8c0b-71d1778c66ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:35.902240 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e5075-1a50-4020-b5dc-4b51aa7efe8b service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:35.912072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a11b1c-1413-42bf-8c0b-71d1778c66ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:35.913471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a11b1c-1413-42bf-8c0b-71d1778c66ce service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:35.954804 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 894/1792] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:35.956974 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f0a58-418f-401a-a42d-0bc5da9c3949 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:35.957276 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f0a58-418f-401a-a42d-0bc5da9c3949 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:35.965217 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f0a58-418f-401a-a42d-0bc5da9c3949 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:35.968387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 895/1793] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 9 23:02:35 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 23:02:36.091026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d942c6-c8fe-4ad4-b05a-a6f1deb0f2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:36.092449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d942c6-c8fe-4ad4-b05a-a6f1deb0f2b4 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:36.092740 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d942c6-c8fe-4ad4-b05a-a6f1deb0f2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:36.092843 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c910c62-334f-4c66-b26a-138d3dc594de None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:36.093356 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c910c62-334f-4c66-b26a-138d3dc594de None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:36.093443 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d942c6-c8fe-4ad4-b05a-a6f1deb0f2b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:36.093513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c910c62-334f-4c66-b26a-138d3dc594de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:36.094546 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d942c6-c8fe-4ad4-b05a-a6f1deb0f2b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:36.096569 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d942c6-c8fe-4ad4-b05a-a6f1deb0f2b4 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:36.117598 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 899/1794] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:36.119007 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d942c6-c8fe-4ad4-b05a-a6f1deb0f2b4 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:36.133386 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d942c6-c8fe-4ad4-b05a-a6f1deb0f2b4 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:36.138822 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-5c910c62-334f-4c66-b26a-138d3dc594de None None] MFA Rules not processed for user `02fcd7cc86134ce488c7dd75dfb174b8`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:36.139727 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca681f75-7774-41c7-b513-9b3b7cfb1a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:36.140063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5c910c62-334f-4c66-b26a-138d3dc594de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:36.141039 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca681f75-7774-41c7-b513-9b3b7cfb1a4d admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:36.142098 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca681f75-7774-41c7-b513-9b3b7cfb1a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:36.142574 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca681f75-7774-41c7-b513-9b3b7cfb1a4d admin admin] PATH_INFO: `/v3/projects/0063cff9077045178b3c5222cd6f1a2d/users/11ac43ebff194133b7440f538eb11e63/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:36.163465 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca681f75-7774-41c7-b513-9b3b7cfb1a4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0063cff9077045178b3c5222cd6f1a2d, user_id=11ac43ebff194133b7440f538eb11e63, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:36.167328 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca681f75-7774-41c7-b513-9b3b7cfb1a4d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:36.192795 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 900/1795] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:36 2022] PUT /identity/v3/projects/0063cff9077045178b3c5222cd6f1a2d/users/11ac43ebff194133b7440f538eb11e63/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:36.194340 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca681f75-7774-41c7-b513-9b3b7cfb1a4d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:36.196642 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ca681f75-7774-41c7-b513-9b3b7cfb1a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:36.381252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 896/1796] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 9 23:02:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:02:36.382422 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c910c62-334f-4c66-b26a-138d3dc594de None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:36.409819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c910c62-334f-4c66-b26a-138d3dc594de None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:36.412846 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca681f75-7774-41c7-b513-9b3b7cfb1a4d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:36.422086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5e8ed-bc91-4009-a02c-9705ed9dd144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:36.423489 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5e8ed-bc91-4009-a02c-9705ed9dd144 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:36.423917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5e8ed-bc91-4009-a02c-9705ed9dd144 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:36.424434 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5e8ed-bc91-4009-a02c-9705ed9dd144 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:36.425639 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5e8ed-bc91-4009-a02c-9705ed9dd144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:36.427410 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5e8ed-bc91-4009-a02c-9705ed9dd144 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:36.432267 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb45518-75cc-42c4-a323-3046155222fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:36.434631 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb45518-75cc-42c4-a323-3046155222fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:36.434913 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb45518-75cc-42c4-a323-3046155222fb admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:36.435469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb45518-75cc-42c4-a323-3046155222fb admin admin] PATH_INFO: `/v3/users/619b722988f74c0fab61b0486efb1a21` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:36.439569 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb45518-75cc-42c4-a323-3046155222fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=619b722988f74c0fab61b0486efb1a21)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:36.450199 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb45518-75cc-42c4-a323-3046155222fb admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:36.453200 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 897/1797] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:36.457319 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5e8ed-bc91-4009-a02c-9705ed9dd144 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:36.476524 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5e8ed-bc91-4009-a02c-9705ed9dd144 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:36.487000 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb7164-f229-4d79-b6e4-a2c6d4a660cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:36.491184 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb7164-f229-4d79-b6e4-a2c6d4a660cd admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:36.491752 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb7164-f229-4d79-b6e4-a2c6d4a660cd admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:36.492215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb7164-f229-4d79-b6e4-a2c6d4a660cd admin admin] PATH_INFO: `/v3/projects/0063cff9077045178b3c5222cd6f1a2d/users/11ac43ebff194133b7440f538eb11e63/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:36.498434 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb7164-f229-4d79-b6e4-a2c6d4a660cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0063cff9077045178b3c5222cd6f1a2d, user_id=11ac43ebff194133b7440f538eb11e63, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:36.503999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb7164-f229-4d79-b6e4-a2c6d4a660cd admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:36.533576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 898/1798] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:36 2022] PUT /identity/v3/projects/0063cff9077045178b3c5222cd6f1a2d/users/11ac43ebff194133b7440f538eb11e63/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:36.545305 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-3bb45518-75cc-42c4-a323-3046155222fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '619b722988f74c0fab61b0486efb1a21'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:36.547080 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76931968-8469-4d7e-8f9c-fea4f4126458 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:36.548273 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76931968-8469-4d7e-8f9c-fea4f4126458 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:36.548798 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76931968-8469-4d7e-8f9c-fea4f4126458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:36.554903 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-3bb45518-75cc-42c4-a323-3046155222fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '619b722988f74c0fab61b0486efb1a21'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:36.567657 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-3bb45518-75cc-42c4-a323-3046155222fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '619b722988f74c0fab61b0486efb1a21'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:36.568439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-3bb45518-75cc-42c4-a323-3046155222fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '619b722988f74c0fab61b0486efb1a21'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:36.606826 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-76931968-8469-4d7e-8f9c-fea4f4126458 None None] MFA Rules not processed for user `11ac43ebff194133b7440f538eb11e63`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:36.608425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-76931968-8469-4d7e-8f9c-fea4f4126458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:36.611565 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-3bb45518-75cc-42c4-a323-3046155222fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '619b722988f74c0fab61b0486efb1a21'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:36.623451 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 901/1799] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:36 2022] DELETE /identity/v3/users/619b722988f74c0fab61b0486efb1a21 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:36.630805 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b1ce3-5b11-4260-be60-ff1bad777a08 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:36.631093 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b1ce3-5b11-4260-be60-ff1bad777a08 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:36.631372 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b1ce3-5b11-4260-be60-ff1bad777a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:36.674012 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-125b1ce3-5b11-4260-be60-ff1bad777a08 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:36.679149 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-125b1ce3-5b11-4260-be60-ff1bad777a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:36.819447 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 899/1800] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:36.833604 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76931968-8469-4d7e-8f9c-fea4f4126458 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:36.835489 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-76931968-8469-4d7e-8f9c-fea4f4126458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:36.878395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 902/1801] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:02:36.954324 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76931968-8469-4d7e-8f9c-fea4f4126458 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:36.973266 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b1ce3-5b11-4260-be60-ff1bad777a08 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:36.976403 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-125b1ce3-5b11-4260-be60-ff1bad777a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:37.057441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ca163-a2ac-426c-8ced-9cad9bf7736c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:37.057441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ca163-a2ac-426c-8ced-9cad9bf7736c service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:37.057441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ca163-a2ac-426c-8ced-9cad9bf7736c service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:37.057441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ca163-a2ac-426c-8ced-9cad9bf7736c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:37.059142 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b1ce3-5b11-4260-be60-ff1bad777a08 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:37.063399 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477c0e8-0299-4530-9557-8ecf610004bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:37.064172 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477c0e8-0299-4530-9557-8ecf610004bb service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:37.064502 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477c0e8-0299-4530-9557-8ecf610004bb service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:37.064924 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477c0e8-0299-4530-9557-8ecf610004bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:37.075378 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6477c0e8-0299-4530-9557-8ecf610004bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:37.077720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6477c0e8-0299-4530-9557-8ecf610004bb service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:37.093275 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 903/1802] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 23:02:37.137207 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477c0e8-0299-4530-9557-8ecf610004bb service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:37.138758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6477c0e8-0299-4530-9557-8ecf610004bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:37.166602 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477c0e8-0299-4530-9557-8ecf610004bb service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:37.178221 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61036bef-156a-469a-a874-bb17af230f90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:37.178985 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61036bef-156a-469a-a874-bb17af230f90 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:37.179340 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61036bef-156a-469a-a874-bb17af230f90 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:37.180034 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61036bef-156a-469a-a874-bb17af230f90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:37.185278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-61036bef-156a-469a-a874-bb17af230f90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:37.211860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9ca163-a2ac-426c-8ced-9cad9bf7736c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:37.213467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9ca163-a2ac-426c-8ced-9cad9bf7736c service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:37.262686 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 900/1803] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 23:02:37.340665 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61036bef-156a-469a-a874-bb17af230f90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:37.347894 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61036bef-156a-469a-a874-bb17af230f90 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:37.374318 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 904/1804] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 23:02:37.393735 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ca163-a2ac-426c-8ced-9cad9bf7736c service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:37.395534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-6e9ca163-a2ac-426c-8ced-9cad9bf7736c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:37.423446 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ca163-a2ac-426c-8ced-9cad9bf7736c service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:37.434138 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b48c57-8502-4802-85b1-2bb699f2f536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:37.437465 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b48c57-8502-4802-85b1-2bb699f2f536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:37.438463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b48c57-8502-4802-85b1-2bb699f2f536 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:37.438780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b48c57-8502-4802-85b1-2bb699f2f536 admin admin] PATH_INFO: `/v3/users/b4eb72b9a3fc43ceb0177148fda513a4` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:37.444161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b48c57-8502-4802-85b1-2bb699f2f536 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4eb72b9a3fc43ceb0177148fda513a4)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:37.446188 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b48c57-8502-4802-85b1-2bb699f2f536 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:37.553329 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-82b48c57-8502-4802-85b1-2bb699f2f536 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4eb72b9a3fc43ceb0177148fda513a4'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:37.575526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-82b48c57-8502-4802-85b1-2bb699f2f536 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4eb72b9a3fc43ceb0177148fda513a4'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:37.579248 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-82b48c57-8502-4802-85b1-2bb699f2f536 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4eb72b9a3fc43ceb0177148fda513a4'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:37.602359 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-82b48c57-8502-4802-85b1-2bb699f2f536 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4eb72b9a3fc43ceb0177148fda513a4'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:37.603518 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-82b48c57-8502-4802-85b1-2bb699f2f536 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4eb72b9a3fc43ceb0177148fda513a4'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:37.676513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 901/1805] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:37 2022] DELETE /identity/v3/users/b4eb72b9a3fc43ceb0177148fda513a4 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:02:37.681506 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d055970-eac8-450d-950f-4140f9d1b76d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:37.682007 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d055970-eac8-450d-950f-4140f9d1b76d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:37.682275 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d055970-eac8-450d-950f-4140f9d1b76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:37.693257 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b48c57-8502-4802-85b1-2bb699f2f536 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:37.702597 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-82b48c57-8502-4802-85b1-2bb699f2f536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:37.728891 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-7d055970-eac8-450d-950f-4140f9d1b76d None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:37.734722 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7d055970-eac8-450d-950f-4140f9d1b76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:38.001852 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b48c57-8502-4802-85b1-2bb699f2f536 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:38.019639 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 905/1806] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:38.022828 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd3942-d253-4010-8e83-6c49ba90c445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:38.022828 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd3942-d253-4010-8e83-6c49ba90c445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:38.026062 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d055970-eac8-450d-950f-4140f9d1b76d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:38.028913 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd3942-d253-4010-8e83-6c49ba90c445 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:38.030493 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd3942-d253-4010-8e83-6c49ba90c445 admin admin] PATH_INFO: `/v3/projects/a266a38f49b049e1bd5e233b55fad7c6` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:38.040806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd3942-d253-4010-8e83-6c49ba90c445 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a266a38f49b049e1bd5e233b55fad7c6)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:38.041951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7d055970-eac8-450d-950f-4140f9d1b76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:38.044576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd3942-d253-4010-8e83-6c49ba90c445 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:38.101365 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d055970-eac8-450d-950f-4140f9d1b76d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:38.127705 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860e733-286d-4043-9516-4a49d1be633e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:38.131532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860e733-286d-4043-9516-4a49d1be633e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:38.132091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860e733-286d-4043-9516-4a49d1be633e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:38.145465 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860e733-286d-4043-9516-4a49d1be633e admin admin] PATH_INFO: `/v3/projects/9cc44f673423451a96d29d11834c3a29` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:38.159488 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860e733-286d-4043-9516-4a49d1be633e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cc44f673423451a96d29d11834c3a29)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:38.166193 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860e733-286d-4043-9516-4a49d1be633e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:38.285837 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-dafd3942-d253-4010-8e83-6c49ba90c445 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a266a38f49b049e1bd5e233b55fad7c6', 'request_id': 'req-dafd3942-d253-4010-8e83-6c49ba90c445'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:38.313559 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-dafd3942-d253-4010-8e83-6c49ba90c445 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a266a38f49b049e1bd5e233b55fad7c6', 'request_id': 'req-dafd3942-d253-4010-8e83-6c49ba90c445'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:38.336669 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-dafd3942-d253-4010-8e83-6c49ba90c445 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a266a38f49b049e1bd5e233b55fad7c6', 'request_id': 'req-dafd3942-d253-4010-8e83-6c49ba90c445'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:38.364074 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c860e733-286d-4043-9516-4a49d1be633e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cc44f673423451a96d29d11834c3a29', 'request_id': 'req-c860e733-286d-4043-9516-4a49d1be633e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:38.369818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-dafd3942-d253-4010-8e83-6c49ba90c445 admin admin] The token cache is being invalidate because project a266a38f49b049e1bd5e233b55fad7c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:38.371040 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-dafd3942-d253-4010-8e83-6c49ba90c445 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:38.373331 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-dafd3942-d253-4010-8e83-6c49ba90c445 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:38.380667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 902/1807] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:37 2022] DELETE /identity/v3/projects/a266a38f49b049e1bd5e233b55fad7c6 => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:38.380667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd3942-d253-4010-8e83-6c49ba90c445 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:38.383082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c860e733-286d-4043-9516-4a49d1be633e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cc44f673423451a96d29d11834c3a29', 'request_id': 'req-c860e733-286d-4043-9516-4a49d1be633e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:38.391581 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c860e733-286d-4043-9516-4a49d1be633e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cc44f673423451a96d29d11834c3a29', 'request_id': 'req-c860e733-286d-4043-9516-4a49d1be633e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:38.395983 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-dafd3942-d253-4010-8e83-6c49ba90c445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:38.434881 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c860e733-286d-4043-9516-4a49d1be633e admin admin] The token cache is being invalidate because project 9cc44f673423451a96d29d11834c3a29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:38.438975 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c860e733-286d-4043-9516-4a49d1be633e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:38.443175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c860e733-286d-4043-9516-4a49d1be633e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:38.453908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 906/1808] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:38 2022] DELETE /identity/v3/projects/9cc44f673423451a96d29d11834c3a29 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:38.462609 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ba60d-468c-4aa8-ad2b-370d68123e8c None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:38.463001 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ba60d-468c-4aa8-ad2b-370d68123e8c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:38.463423 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ba60d-468c-4aa8-ad2b-370d68123e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:38.580521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd3942-d253-4010-8e83-6c49ba90c445 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:38.616316 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-398ba60d-468c-4aa8-ad2b-370d68123e8c None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:38.628934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-398ba60d-468c-4aa8-ad2b-370d68123e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:38.785013 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85732e6c-822c-4dda-bb9b-f8dda52ff146 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:38.786228 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85732e6c-822c-4dda-bb9b-f8dda52ff146 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:38.787928 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85732e6c-822c-4dda-bb9b-f8dda52ff146 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:38.789590 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85732e6c-822c-4dda-bb9b-f8dda52ff146 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:38.796127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-85732e6c-822c-4dda-bb9b-f8dda52ff146 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:38.838167 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85732e6c-822c-4dda-bb9b-f8dda52ff146 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:38.842943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85732e6c-822c-4dda-bb9b-f8dda52ff146 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:38.881323 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 907/1809] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:38.882242 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 903/1810] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:38.883950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c333566-75dc-408d-ad6a-fca516c05aee None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:38.884464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c333566-75dc-408d-ad6a-fca516c05aee None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:38.885065 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c333566-75dc-408d-ad6a-fca516c05aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:38.885193 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85732e6c-822c-4dda-bb9b-f8dda52ff146 service nova] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:38.887203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-85732e6c-822c-4dda-bb9b-f8dda52ff146 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:38.959734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5c333566-75dc-408d-ad6a-fca516c05aee None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:38.959734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5c333566-75dc-408d-ad6a-fca516c05aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:38.993220 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 908/1811] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:38.997080 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74533d1-e6cc-480c-bbec-d8f078ead38c None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:38.997496 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74533d1-e6cc-480c-bbec-d8f078ead38c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:38.998046 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74533d1-e6cc-480c-bbec-d8f078ead38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:39.067981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-b74533d1-e6cc-480c-bbec-d8f078ead38c None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:39.071481 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b74533d1-e6cc-480c-bbec-d8f078ead38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:39.093984 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 909/1812] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:39.095481 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65c155-372f-4657-901f-7b452ad00bdf None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:39.100272 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65c155-372f-4657-901f-7b452ad00bdf None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:39.100800 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65c155-372f-4657-901f-7b452ad00bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:39.118482 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85732e6c-822c-4dda-bb9b-f8dda52ff146 service nova] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:39.128671 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2e4e6-ff3d-476f-80dd-b51c85f196ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:39.130575 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2e4e6-ff3d-476f-80dd-b51c85f196ed service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:39.130857 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2e4e6-ff3d-476f-80dd-b51c85f196ed service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:39.131130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2e4e6-ff3d-476f-80dd-b51c85f196ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:39.136679 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e1c2e4e6-ff3d-476f-80dd-b51c85f196ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:39.145134 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3d65c155-372f-4657-901f-7b452ad00bdf None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:39.147008 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3d65c155-372f-4657-901f-7b452ad00bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:39.159034 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c2e4e6-ff3d-476f-80dd-b51c85f196ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:39.160522 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c2e4e6-ff3d-476f-80dd-b51c85f196ed service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:39.175208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 910/1813] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:39.176155 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65c155-372f-4657-901f-7b452ad00bdf None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:39.193205 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 904/1814] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:39.194052 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2e4e6-ff3d-476f-80dd-b51c85f196ed service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:39.195767 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65c155-372f-4657-901f-7b452ad00bdf None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:39.210661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2e4e6-ff3d-476f-80dd-b51c85f196ed service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:39.210850 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013dba40-7ed3-48de-84d0-d71e45afa352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:39.211504 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013dba40-7ed3-48de-84d0-d71e45afa352 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:39.211872 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013dba40-7ed3-48de-84d0-d71e45afa352 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:39.212072 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013dba40-7ed3-48de-84d0-d71e45afa352 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:39.212775 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013dba40-7ed3-48de-84d0-d71e45afa352 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:02:39.213225 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013dba40-7ed3-48de-84d0-d71e45afa352 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:39.214890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013dba40-7ed3-48de-84d0-d71e45afa352 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:39.228179 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099e063-2b08-40c7-ad9f-9c8394840b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:39.228816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099e063-2b08-40c7-ad9f-9c8394840b12 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:39.229095 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099e063-2b08-40c7-ad9f-9c8394840b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:39.229368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099e063-2b08-40c7-ad9f-9c8394840b12 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:39.230092 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7099e063-2b08-40c7-ad9f-9c8394840b12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:02:39.230417 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7099e063-2b08-40c7-ad9f-9c8394840b12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:39.233978 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7099e063-2b08-40c7-ad9f-9c8394840b12 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:39.240623 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 911/1815] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:02:39.249252 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013dba40-7ed3-48de-84d0-d71e45afa352 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:39.271346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013dba40-7ed3-48de-84d0-d71e45afa352 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:39.277574 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81d4dd-00ce-4511-ae02-520e7fee3c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:39.278354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81d4dd-00ce-4511-ae02-520e7fee3c20 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:39.278639 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81d4dd-00ce-4511-ae02-520e7fee3c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:39.278914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81d4dd-00ce-4511-ae02-520e7fee3c20 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:39.279615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81d4dd-00ce-4511-ae02-520e7fee3c20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:39.280560 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 905/1816] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:02:39.282496 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81d4dd-00ce-4511-ae02-520e7fee3c20 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:39.288120 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099e063-2b08-40c7-ad9f-9c8394840b12 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:39.290426 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7099e063-2b08-40c7-ad9f-9c8394840b12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:39.332079 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 912/1817] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:39 2022] POST /identity/v3/projects => generated 379 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:39.333090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81d4dd-00ce-4511-ae02-520e7fee3c20 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:39.337698 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099e063-2b08-40c7-ad9f-9c8394840b12 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:39.467985 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81d4dd-00ce-4511-ae02-520e7fee3c20 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:39.473996 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79e2c1-57a2-4807-91cd-687086f9693a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:39.475226 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79e2c1-57a2-4807-91cd-687086f9693a admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:39.475615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79e2c1-57a2-4807-91cd-687086f9693a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:39.475996 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79e2c1-57a2-4807-91cd-687086f9693a admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:39.476852 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff79e2c1-57a2-4807-91cd-687086f9693a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:39.479086 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff79e2c1-57a2-4807-91cd-687086f9693a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:39.533917 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 913/1818] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:39 2022] POST /identity/v3/projects => generated 375 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:39.534976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79e2c1-57a2-4807-91cd-687086f9693a admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:39.574753 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7709ffb-a1a1-43c8-b474-d5c64555823d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:39.578030 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7709ffb-a1a1-43c8-b474-d5c64555823d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:39.578591 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7709ffb-a1a1-43c8-b474-d5c64555823d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:39.584293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7709ffb-a1a1-43c8-b474-d5c64555823d admin admin] PATH_INFO: `/v3/projects/c19350cdee8a47a7835ff7727d949f60` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:39.588377 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7709ffb-a1a1-43c8-b474-d5c64555823d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c19350cdee8a47a7835ff7727d949f60)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:39.591041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7709ffb-a1a1-43c8-b474-d5c64555823d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:39.721755 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79e2c1-57a2-4807-91cd-687086f9693a admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:39.761812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d7709ffb-a1a1-43c8-b474-d5c64555823d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c19350cdee8a47a7835ff7727d949f60', 'request_id': 'req-d7709ffb-a1a1-43c8-b474-d5c64555823d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:39.795242 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d7709ffb-a1a1-43c8-b474-d5c64555823d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c19350cdee8a47a7835ff7727d949f60', 'request_id': 'req-d7709ffb-a1a1-43c8-b474-d5c64555823d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:39.813423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d7709ffb-a1a1-43c8-b474-d5c64555823d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c19350cdee8a47a7835ff7727d949f60', 'request_id': 'req-d7709ffb-a1a1-43c8-b474-d5c64555823d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:39.861437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d7709ffb-a1a1-43c8-b474-d5c64555823d admin admin] The token cache is being invalidate because project c19350cdee8a47a7835ff7727d949f60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:39.868421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d7709ffb-a1a1-43c8-b474-d5c64555823d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:39.870715 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557fdfaa-0b5a-4519-9108-f70312b456bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:39.872011 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557fdfaa-0b5a-4519-9108-f70312b456bd admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:39.874399 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557fdfaa-0b5a-4519-9108-f70312b456bd admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:39.874763 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d7709ffb-a1a1-43c8-b474-d5c64555823d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:39.875091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557fdfaa-0b5a-4519-9108-f70312b456bd admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:39.878637 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557fdfaa-0b5a-4519-9108-f70312b456bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:39.881656 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 906/1819] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:39 2022] DELETE /identity/v3/projects/c19350cdee8a47a7835ff7727d949f60 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:02:39.883768 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557fdfaa-0b5a-4519-9108-f70312b456bd admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:39.890008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7709ffb-a1a1-43c8-b474-d5c64555823d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:39.899727 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d7709ffb-a1a1-43c8-b474-d5c64555823d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:39.931273 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7709ffb-a1a1-43c8-b474-d5c64555823d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:39.947456 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6191e3-c0b3-4ae7-ad75-7c3581c26ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:39.948826 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6191e3-c0b3-4ae7-ad75-7c3581c26ac7 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:39.949254 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6191e3-c0b3-4ae7-ad75-7c3581c26ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:39.949586 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6191e3-c0b3-4ae7-ad75-7c3581c26ac7 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:39.949932 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 914/1820] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:39 2022] POST /identity/v3/users => generated 290 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:39.951374 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6191e3-c0b3-4ae7-ad75-7c3581c26ac7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:39.953761 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6191e3-c0b3-4ae7-ad75-7c3581c26ac7 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:39.954814 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557fdfaa-0b5a-4519-9108-f70312b456bd admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:39.961592 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-557fdfaa-0b5a-4519-9108-f70312b456bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:39.996357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 907/1821] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:39 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:39.997309 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6191e3-c0b3-4ae7-ad75-7c3581c26ac7 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:39.998460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557fdfaa-0b5a-4519-9108-f70312b456bd admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:40.001483 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-bd6191e3-c0b3-4ae7-ad75-7c3581c26ac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:40.010609 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f88ac1-c7f9-496d-887a-b26d9beef5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:40.011964 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f88ac1-c7f9-496d-887a-b26d9beef5b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:40.014598 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f88ac1-c7f9-496d-887a-b26d9beef5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:40.014986 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f88ac1-c7f9-496d-887a-b26d9beef5b6 admin admin] PATH_INFO: `/v3/users/ba2897016aaf42a1a4c1115c58f4f80a` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:40.017914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f88ac1-c7f9-496d-887a-b26d9beef5b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba2897016aaf42a1a4c1115c58f4f80a)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:40.022381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f88ac1-c7f9-496d-887a-b26d9beef5b6 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:40.075286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6191e3-c0b3-4ae7-ad75-7c3581c26ac7 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:40.098257 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf95fd-d5d9-4b0a-b38f-3a8da0d9d64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:40.101774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf95fd-d5d9-4b0a-b38f-3a8da0d9d64c admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:40.102974 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf95fd-d5d9-4b0a-b38f-3a8da0d9d64c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:40.103693 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf95fd-d5d9-4b0a-b38f-3a8da0d9d64c admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:40.107949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf95fd-d5d9-4b0a-b38f-3a8da0d9d64c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:40.113694 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf95fd-d5d9-4b0a-b38f-3a8da0d9d64c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:40.130626 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 908/1822] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:40.132136 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf95fd-d5d9-4b0a-b38f-3a8da0d9d64c admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:40.155101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf95fd-d5d9-4b0a-b38f-3a8da0d9d64c admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:40.157805 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f4f88ac1-c7f9-496d-887a-b26d9beef5b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba2897016aaf42a1a4c1115c58f4f80a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:40.178180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b5bbd-8d1f-4447-aa64-ceed6e3774ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:40.183822 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f4f88ac1-c7f9-496d-887a-b26d9beef5b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba2897016aaf42a1a4c1115c58f4f80a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:40.192173 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b5bbd-8d1f-4447-aa64-ceed6e3774ff admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:40.192369 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b5bbd-8d1f-4447-aa64-ceed6e3774ff admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:40.192600 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b5bbd-8d1f-4447-aa64-ceed6e3774ff admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:40.195086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b5bbd-8d1f-4447-aa64-ceed6e3774ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:40.197670 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b5bbd-8d1f-4447-aa64-ceed6e3774ff admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:40.217313 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f4f88ac1-c7f9-496d-887a-b26d9beef5b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba2897016aaf42a1a4c1115c58f4f80a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:40.219989 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f4f88ac1-c7f9-496d-887a-b26d9beef5b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba2897016aaf42a1a4c1115c58f4f80a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:40.231597 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 909/1823] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:40.233059 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b5bbd-8d1f-4447-aa64-ceed6e3774ff admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:40.266842 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f4f88ac1-c7f9-496d-887a-b26d9beef5b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba2897016aaf42a1a4c1115c58f4f80a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:40.276135 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 915/1824] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:39 2022] DELETE /identity/v3/users/ba2897016aaf42a1a4c1115c58f4f80a => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:40.276983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f88ac1-c7f9-496d-887a-b26d9beef5b6 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:40.279458 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f4f88ac1-c7f9-496d-887a-b26d9beef5b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:40.282957 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b5bbd-8d1f-4447-aa64-ceed6e3774ff admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:40.437197 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f88ac1-c7f9-496d-887a-b26d9beef5b6 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:40.439292 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3a460-48f8-490c-b3d0-092bf8fc0eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:40.446831 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3a460-48f8-490c-b3d0-092bf8fc0eaa admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:40.447467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3a460-48f8-490c-b3d0-092bf8fc0eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:40.447806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3a460-48f8-490c-b3d0-092bf8fc0eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:40.448769 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3a460-48f8-490c-b3d0-092bf8fc0eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:40.450781 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3a460-48f8-490c-b3d0-092bf8fc0eaa admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:40.453579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264d41d-d138-4910-921a-1c1b74b7f27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:40.454289 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264d41d-d138-4910-921a-1c1b74b7f27f admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:40.454568 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264d41d-d138-4910-921a-1c1b74b7f27f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:40.454857 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264d41d-d138-4910-921a-1c1b74b7f27f admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:40.455570 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b264d41d-d138-4910-921a-1c1b74b7f27f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:40.457182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b264d41d-d138-4910-921a-1c1b74b7f27f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:40.465829 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 910/1825] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:40.465829 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3a460-48f8-490c-b3d0-092bf8fc0eaa admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:40.465829 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f9c3a460-48f8-490c-b3d0-092bf8fc0eaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:40.478189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 916/1826] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:40.479091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264d41d-d138-4910-921a-1c1b74b7f27f admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:40.481029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b264d41d-d138-4910-921a-1c1b74b7f27f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:40.485130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3a460-48f8-490c-b3d0-092bf8fc0eaa admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:40.492422 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93631ac6-dbe4-4580-898d-22570a763d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:40.493678 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93631ac6-dbe4-4580-898d-22570a763d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:40.493962 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93631ac6-dbe4-4580-898d-22570a763d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:40.494270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93631ac6-dbe4-4580-898d-22570a763d5a admin admin] PATH_INFO: `/v3/projects/f32e8158864e449b9fd607276092ef3f` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:40.495480 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93631ac6-dbe4-4580-898d-22570a763d5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f32e8158864e449b9fd607276092ef3f)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:40.513052 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264d41d-d138-4910-921a-1c1b74b7f27f admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:40.517776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c2424-4db3-4bac-8cda-1940241944ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:40.518807 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c2424-4db3-4bac-8cda-1940241944ac admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:40.519174 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c2424-4db3-4bac-8cda-1940241944ac admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:40.519644 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c2424-4db3-4bac-8cda-1940241944ac admin admin] PATH_INFO: `/v3/projects/6ae66866abe84d6596765919b1f546dd/users/4b51b0be36c24568aab90516e3c29916/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:40.527031 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93631ac6-dbe4-4580-898d-22570a763d5a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:40.541025 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32c2424-4db3-4bac-8cda-1940241944ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ae66866abe84d6596765919b1f546dd, user_id=4b51b0be36c24568aab90516e3c29916, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:40.544979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32c2424-4db3-4bac-8cda-1940241944ac admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:40.574164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 917/1827] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:40 2022] PUT /identity/v3/projects/6ae66866abe84d6596765919b1f546dd/users/4b51b0be36c24568aab90516e3c29916/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:40.574164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c2424-4db3-4bac-8cda-1940241944ac admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:40.648510 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-93631ac6-dbe4-4580-898d-22570a763d5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f32e8158864e449b9fd607276092ef3f', 'request_id': 'req-93631ac6-dbe4-4580-898d-22570a763d5a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:40.664806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-93631ac6-dbe4-4580-898d-22570a763d5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f32e8158864e449b9fd607276092ef3f', 'request_id': 'req-93631ac6-dbe4-4580-898d-22570a763d5a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:40.681777 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-93631ac6-dbe4-4580-898d-22570a763d5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f32e8158864e449b9fd607276092ef3f', 'request_id': 'req-93631ac6-dbe4-4580-898d-22570a763d5a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:40.706815 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-93631ac6-dbe4-4580-898d-22570a763d5a admin admin] The token cache is being invalidate because project f32e8158864e449b9fd607276092ef3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:40.707669 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-93631ac6-dbe4-4580-898d-22570a763d5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:40.709236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-93631ac6-dbe4-4580-898d-22570a763d5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:40.712415 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 911/1828] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:40 2022] DELETE /identity/v3/projects/f32e8158864e449b9fd607276092ef3f => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:40.714635 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2119a89-d7b4-44cd-bb9c-f5e429b09fe9 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:40.715049 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2119a89-d7b4-44cd-bb9c-f5e429b09fe9 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:40.715568 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2119a89-d7b4-44cd-bb9c-f5e429b09fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:40.759556 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b2119a89-d7b4-44cd-bb9c-f5e429b09fe9 None None] MFA Rules not processed for user `4b51b0be36c24568aab90516e3c29916`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:40.761632 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b2119a89-d7b4-44cd-bb9c-f5e429b09fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:40.780820 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c2424-4db3-4bac-8cda-1940241944ac admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:40.850304 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 912/1829] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:40.855971 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2119a89-d7b4-44cd-bb9c-f5e429b09fe9 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:40.858304 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b2119a89-d7b4-44cd-bb9c-f5e429b09fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:40.956173 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb3228-dcca-4bc8-a698-d5e546362220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:40.962653 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb3228-dcca-4bc8-a698-d5e546362220 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:40.963898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb3228-dcca-4bc8-a698-d5e546362220 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:40.964327 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb3228-dcca-4bc8-a698-d5e546362220 admin admin] PATH_INFO: `/v3/projects/4200d3d033234934a42cc5110f10f278/users/2714b23e7c2b4eceb3c9dedcd04fca27/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:40.986720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb3228-dcca-4bc8-a698-d5e546362220 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4200d3d033234934a42cc5110f10f278, user_id=2714b23e7c2b4eceb3c9dedcd04fca27, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:40.997805 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2119a89-d7b4-44cd-bb9c-f5e429b09fe9 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:41.002734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb3228-dcca-4bc8-a698-d5e546362220 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:41.008954 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a7727-65d2-49ff-8a19-bbe936baf611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:41.011060 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a7727-65d2-49ff-8a19-bbe936baf611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:41.012276 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a7727-65d2-49ff-8a19-bbe936baf611 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:41.012552 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a7727-65d2-49ff-8a19-bbe936baf611 admin admin] PATH_INFO: `/v3/users/11ac43ebff194133b7440f538eb11e63` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:41.015330 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a7727-65d2-49ff-8a19-bbe936baf611 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11ac43ebff194133b7440f538eb11e63)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:41.020579 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a7727-65d2-49ff-8a19-bbe936baf611 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:41.030197 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 918/1830] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:40 2022] PUT /identity/v3/projects/4200d3d033234934a42cc5110f10f278/users/2714b23e7c2b4eceb3c9dedcd04fca27/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:41.031695 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb3228-dcca-4bc8-a698-d5e546362220 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:41.033450 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-bfdb3228-dcca-4bc8-a698-d5e546362220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:41.114717 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-777a7727-65d2-49ff-8a19-bbe936baf611 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11ac43ebff194133b7440f538eb11e63'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:41.134526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-777a7727-65d2-49ff-8a19-bbe936baf611 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11ac43ebff194133b7440f538eb11e63'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:41.135934 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-777a7727-65d2-49ff-8a19-bbe936baf611 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11ac43ebff194133b7440f538eb11e63'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:41.152741 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-777a7727-65d2-49ff-8a19-bbe936baf611 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11ac43ebff194133b7440f538eb11e63'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:41.155050 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-777a7727-65d2-49ff-8a19-bbe936baf611 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11ac43ebff194133b7440f538eb11e63'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:41.180307 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb3228-dcca-4bc8-a698-d5e546362220 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:41.192679 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3ff32-7460-473c-bd0f-ebeb3681dd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:41.194149 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3ff32-7460-473c-bd0f-ebeb3681dd99 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:41.194588 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3ff32-7460-473c-bd0f-ebeb3681dd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:41.195066 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3ff32-7460-473c-bd0f-ebeb3681dd99 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:41.196432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3ff32-7460-473c-bd0f-ebeb3681dd99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:41.198382 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 913/1831] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:40 2022] DELETE /identity/v3/users/11ac43ebff194133b7440f538eb11e63 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:41.200164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3ff32-7460-473c-bd0f-ebeb3681dd99 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:41.200289 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff8c98-fa9e-4ca6-8097-4eb698f2c3be None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:41.200516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff8c98-fa9e-4ca6-8097-4eb698f2c3be None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:41.202925 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff8c98-fa9e-4ca6-8097-4eb698f2c3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:41.267045 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-94ff8c98-fa9e-4ca6-8097-4eb698f2c3be None None] MFA Rules not processed for user `2714b23e7c2b4eceb3c9dedcd04fca27`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:41.268563 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-94ff8c98-fa9e-4ca6-8097-4eb698f2c3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:41.302142 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 919/1832] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:41 2022] POST /identity/v3/projects => generated 377 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:41.305443 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3ff32-7460-473c-bd0f-ebeb3681dd99 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:41.313872 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f9c3ff32-7460-473c-bd0f-ebeb3681dd99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:41.448809 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 914/1833] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:41.453977 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff8c98-fa9e-4ca6-8097-4eb698f2c3be None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:41.457153 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-94ff8c98-fa9e-4ca6-8097-4eb698f2c3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:41.578972 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3ff32-7460-473c-bd0f-ebeb3681dd99 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:41.595683 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e38a98-5601-42bb-b7ee-2b8b63f87b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:41.600003 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e38a98-5601-42bb-b7ee-2b8b63f87b03 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:41.600301 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e38a98-5601-42bb-b7ee-2b8b63f87b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:41.601401 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e38a98-5601-42bb-b7ee-2b8b63f87b03 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:41.603782 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e38a98-5601-42bb-b7ee-2b8b63f87b03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:41.607007 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e38a98-5601-42bb-b7ee-2b8b63f87b03 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:41.677185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff8c98-fa9e-4ca6-8097-4eb698f2c3be None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:41.682852 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 920/1834] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:41 2022] POST /identity/v3/users => generated 289 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:41.684151 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e38a98-5601-42bb-b7ee-2b8b63f87b03 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:41.688408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-86e38a98-5601-42bb-b7ee-2b8b63f87b03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:41.706739 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddebf8e-b34d-4c93-aada-cfae2150b04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:41.707479 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddebf8e-b34d-4c93-aada-cfae2150b04e admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:41.707728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e38a98-5601-42bb-b7ee-2b8b63f87b03 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:41.707987 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddebf8e-b34d-4c93-aada-cfae2150b04e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:41.707987 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddebf8e-b34d-4c93-aada-cfae2150b04e admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:41.709875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddebf8e-b34d-4c93-aada-cfae2150b04e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:41.714771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012c445-4749-4b9b-9f3c-41d937afb5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:41.715295 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddebf8e-b34d-4c93-aada-cfae2150b04e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:41.716669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012c445-4749-4b9b-9f3c-41d937afb5a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:41.717383 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012c445-4749-4b9b-9f3c-41d937afb5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:41.718138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012c445-4749-4b9b-9f3c-41d937afb5a0 admin admin] PATH_INFO: `/v3/projects/b4705e673a464496ab056abbae151fcf` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:41.719997 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012c445-4749-4b9b-9f3c-41d937afb5a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4705e673a464496ab056abbae151fcf)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:41.724253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012c445-4749-4b9b-9f3c-41d937afb5a0 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:41.817414 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 915/1835] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:41 2022] POST /identity/v3/projects => generated 375 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:41.817414 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddebf8e-b34d-4c93-aada-cfae2150b04e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:41.863813 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9012c445-4749-4b9b-9f3c-41d937afb5a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4705e673a464496ab056abbae151fcf', 'request_id': 'req-9012c445-4749-4b9b-9f3c-41d937afb5a0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:41.886686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9012c445-4749-4b9b-9f3c-41d937afb5a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4705e673a464496ab056abbae151fcf', 'request_id': 'req-9012c445-4749-4b9b-9f3c-41d937afb5a0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:41.903385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9012c445-4749-4b9b-9f3c-41d937afb5a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4705e673a464496ab056abbae151fcf', 'request_id': 'req-9012c445-4749-4b9b-9f3c-41d937afb5a0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:41.983316 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddebf8e-b34d-4c93-aada-cfae2150b04e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:41.991147 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9012c445-4749-4b9b-9f3c-41d937afb5a0 admin admin] The token cache is being invalidate because project b4705e673a464496ab056abbae151fcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:41.991886 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9012c445-4749-4b9b-9f3c-41d937afb5a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:41.992965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9012c445-4749-4b9b-9f3c-41d937afb5a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:41.995261 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 921/1836] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:41 2022] DELETE /identity/v3/projects/b4705e673a464496ab056abbae151fcf => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:41.997033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012c445-4749-4b9b-9f3c-41d937afb5a0 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:42.004652 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9012c445-4749-4b9b-9f3c-41d937afb5a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:42.105563 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17d447-761e-4336-bb03-40af85301eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:42.106350 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17d447-761e-4336-bb03-40af85301eca admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:42.106702 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17d447-761e-4336-bb03-40af85301eca admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:42.107041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17d447-761e-4336-bb03-40af85301eca admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:42.108109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca17d447-761e-4336-bb03-40af85301eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:42.110370 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca17d447-761e-4336-bb03-40af85301eca admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:42.134465 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 916/1837] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:42.137683 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d3c3f-aa4f-434e-a7f6-d8d4a606b3fd None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:42.137984 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d3c3f-aa4f-434e-a7f6-d8d4a606b3fd None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:42.138267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d3c3f-aa4f-434e-a7f6-d8d4a606b3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:42.178348 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-237d3c3f-aa4f-434e-a7f6-d8d4a606b3fd None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:42.179853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-237d3c3f-aa4f-434e-a7f6-d8d4a606b3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:42.189783 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 917/1838] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:42.190764 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d3c3f-aa4f-434e-a7f6-d8d4a606b3fd None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:42.192724 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-237d3c3f-aa4f-434e-a7f6-d8d4a606b3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:42.205918 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d3c3f-aa4f-434e-a7f6-d8d4a606b3fd None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:42.212944 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee01afd-7f31-4571-ad21-ce0b150f51c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:42.213852 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee01afd-7f31-4571-ad21-ce0b150f51c0 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:42.214316 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee01afd-7f31-4571-ad21-ce0b150f51c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:42.220559 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee01afd-7f31-4571-ad21-ce0b150f51c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:42.221537 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee01afd-7f31-4571-ad21-ce0b150f51c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:42.223385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee01afd-7f31-4571-ad21-ce0b150f51c0 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:42.245761 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 918/1839] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:42.247851 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c7073-0737-4b58-9e03-383d1c292277 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:42.250977 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c7073-0737-4b58-9e03-383d1c292277 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:42.251759 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c7073-0737-4b58-9e03-383d1c292277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:42.274936 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012c445-4749-4b9b-9f3c-41d937afb5a0 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:42.277891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-580c7073-0737-4b58-9e03-383d1c292277 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:42.279368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-580c7073-0737-4b58-9e03-383d1c292277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:42.291551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 919/1840] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:42.292702 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c7073-0737-4b58-9e03-383d1c292277 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:42.303990 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d818d-ca8c-44d4-aca2-39696d12ca15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:42.304686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d818d-ca8c-44d4-aca2-39696d12ca15 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:42.305794 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d818d-ca8c-44d4-aca2-39696d12ca15 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:42.306086 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d818d-ca8c-44d4-aca2-39696d12ca15 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:42.306789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c7073-0737-4b58-9e03-383d1c292277 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:42.309210 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d818d-ca8c-44d4-aca2-39696d12ca15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:42.317840 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8a244-4a0c-4abe-b45e-3a9a56a9a359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:42.318136 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d818d-ca8c-44d4-aca2-39696d12ca15 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:42.319452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8a244-4a0c-4abe-b45e-3a9a56a9a359 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:42.319982 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8a244-4a0c-4abe-b45e-3a9a56a9a359 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:42.321521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8a244-4a0c-4abe-b45e-3a9a56a9a359 admin admin] PATH_INFO: `/v3/projects/1f4da2a0d8554ce69aebe1f016dcfe45/users/cdf2ab2b9a734a31bc7e0e194c267dde/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:42.340162 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d8a244-4a0c-4abe-b45e-3a9a56a9a359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f4da2a0d8554ce69aebe1f016dcfe45, user_id=cdf2ab2b9a734a31bc7e0e194c267dde, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:42.343438 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d8a244-4a0c-4abe-b45e-3a9a56a9a359 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:42.385580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 920/1841] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:42 2022] PUT /identity/v3/projects/1f4da2a0d8554ce69aebe1f016dcfe45/users/cdf2ab2b9a734a31bc7e0e194c267dde/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:42.385580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8a244-4a0c-4abe-b45e-3a9a56a9a359 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:42.403070 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 922/1842] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:41 2022] POST /identity/v3/users => generated 294 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:42.403070 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa2dbf-52c8-4b85-b052-cb73ed43199d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:42.403070 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa2dbf-52c8-4b85-b052-cb73ed43199d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:42.413076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa2dbf-52c8-4b85-b052-cb73ed43199d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:42.483046 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-9caa2dbf-52c8-4b85-b052-cb73ed43199d None None] MFA Rules not processed for user `cdf2ab2b9a734a31bc7e0e194c267dde`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:42.486944 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9caa2dbf-52c8-4b85-b052-cb73ed43199d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:42.508231 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8a244-4a0c-4abe-b45e-3a9a56a9a359 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:42.514877 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd17e3-d266-4ab5-b749-e0c569ca450a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:42.516073 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd17e3-d266-4ab5-b749-e0c569ca450a admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:42.516451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd17e3-d266-4ab5-b749-e0c569ca450a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:42.516782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd17e3-d266-4ab5-b749-e0c569ca450a admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:42.517691 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd17e3-d266-4ab5-b749-e0c569ca450a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:02:42.518436 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd17e3-d266-4ab5-b749-e0c569ca450a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:42.520177 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd17e3-d266-4ab5-b749-e0c569ca450a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:42.540753 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 921/1843] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:02:42.540753 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd17e3-d266-4ab5-b749-e0c569ca450a admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:42.562972 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd17e3-d266-4ab5-b749-e0c569ca450a admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:42.574993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fae30-b4d3-4c74-836f-71719adc4d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:42.576966 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fae30-b4d3-4c74-836f-71719adc4d7d admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:42.578774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fae30-b4d3-4c74-836f-71719adc4d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:42.579141 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fae30-b4d3-4c74-836f-71719adc4d7d admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:42.579957 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6fae30-b4d3-4c74-836f-71719adc4d7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:42.587047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6fae30-b4d3-4c74-836f-71719adc4d7d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:42.625849 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 922/1844] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:42.625849 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fae30-b4d3-4c74-836f-71719adc4d7d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:42.659384 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fae30-b4d3-4c74-836f-71719adc4d7d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:42.667707 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 923/1845] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:42.668471 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa2dbf-52c8-4b85-b052-cb73ed43199d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:42.681677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9caa2dbf-52c8-4b85-b052-cb73ed43199d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:42.707055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e77a0ea-fabe-489a-b4f1-cb70aa553b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:42.711995 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e77a0ea-fabe-489a-b4f1-cb70aa553b27 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:42.715045 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e77a0ea-fabe-489a-b4f1-cb70aa553b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:42.715422 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e77a0ea-fabe-489a-b4f1-cb70aa553b27 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:42.738274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e77a0ea-fabe-489a-b4f1-cb70aa553b27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:42.752535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e77a0ea-fabe-489a-b4f1-cb70aa553b27 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:42.834987 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa2dbf-52c8-4b85-b052-cb73ed43199d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:42.862808 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fe559-bff5-41e1-9da4-0517fcaeb3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:42.864704 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fe559-bff5-41e1-9da4-0517fcaeb3b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:42.865721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fe559-bff5-41e1-9da4-0517fcaeb3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:42.866005 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fe559-bff5-41e1-9da4-0517fcaeb3b9 admin admin] PATH_INFO: `/v3/projects/0063cff9077045178b3c5222cd6f1a2d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:42.867902 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3fe559-bff5-41e1-9da4-0517fcaeb3b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0063cff9077045178b3c5222cd6f1a2d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:42.877160 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3fe559-bff5-41e1-9da4-0517fcaeb3b9 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:42.909658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 923/1846] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:42 2022] POST /identity/v3/projects => generated 383 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:42.915138 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e77a0ea-fabe-489a-b4f1-cb70aa553b27 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:42.999447 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e77a0ea-fabe-489a-b4f1-cb70aa553b27 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:43.006945 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ce3fe559-bff5-41e1-9da4-0517fcaeb3b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0063cff9077045178b3c5222cd6f1a2d', 'request_id': 'req-ce3fe559-bff5-41e1-9da4-0517fcaeb3b9'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:43.031239 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ce3fe559-bff5-41e1-9da4-0517fcaeb3b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0063cff9077045178b3c5222cd6f1a2d', 'request_id': 'req-ce3fe559-bff5-41e1-9da4-0517fcaeb3b9'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:43.054821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ce3fe559-bff5-41e1-9da4-0517fcaeb3b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0063cff9077045178b3c5222cd6f1a2d', 'request_id': 'req-ce3fe559-bff5-41e1-9da4-0517fcaeb3b9'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:43.074296 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07084ad-3435-4b51-91ce-c217f5f8a4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:43.075622 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07084ad-3435-4b51-91ce-c217f5f8a4bb admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:43.075892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07084ad-3435-4b51-91ce-c217f5f8a4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:43.076175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07084ad-3435-4b51-91ce-c217f5f8a4bb admin admin] PATH_INFO: `/v3/projects/dc181a40b42a4f51982126b1bcbfebb2/users/94e7ebc9886c44d387f701e2a3e1dab3/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:43.087756 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ce3fe559-bff5-41e1-9da4-0517fcaeb3b9 admin admin] The token cache is being invalidate because project 0063cff9077045178b3c5222cd6f1a2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:43.087965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ce3fe559-bff5-41e1-9da4-0517fcaeb3b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:43.089163 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ce3fe559-bff5-41e1-9da4-0517fcaeb3b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:43.091447 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 924/1847] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:42 2022] DELETE /identity/v3/projects/0063cff9077045178b3c5222cd6f1a2d => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:43.092486 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fe559-bff5-41e1-9da4-0517fcaeb3b9 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:43.097653 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ce3fe559-bff5-41e1-9da4-0517fcaeb3b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:43.120929 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07084ad-3435-4b51-91ce-c217f5f8a4bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc181a40b42a4f51982126b1bcbfebb2, user_id=94e7ebc9886c44d387f701e2a3e1dab3, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:43.123750 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07084ad-3435-4b51-91ce-c217f5f8a4bb admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:43.126556 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fe559-bff5-41e1-9da4-0517fcaeb3b9 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:43.134556 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd25e7-e745-4a10-8285-599911271c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:43.135291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd25e7-e745-4a10-8285-599911271c34 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:43.135623 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd25e7-e745-4a10-8285-599911271c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:43.135936 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd25e7-e745-4a10-8285-599911271c34 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:43.136621 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd25e7-e745-4a10-8285-599911271c34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:43.138632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd25e7-e745-4a10-8285-599911271c34 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:43.151823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 924/1848] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:42 2022] PUT /identity/v3/projects/dc181a40b42a4f51982126b1bcbfebb2/users/94e7ebc9886c44d387f701e2a3e1dab3/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:43.152999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07084ad-3435-4b51-91ce-c217f5f8a4bb admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:43.155416 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c07084ad-3435-4b51-91ce-c217f5f8a4bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:43.249420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 925/1849] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:43 2022] POST /identity/v3/projects => generated 377 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:43.251692 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee36f03-2c3b-4f1d-b2a5-aad0c2c351ab None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:43.251953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee36f03-2c3b-4f1d-b2a5-aad0c2c351ab None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:43.252251 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee36f03-2c3b-4f1d-b2a5-aad0c2c351ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:43.271574 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07084ad-3435-4b51-91ce-c217f5f8a4bb admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:43.339698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-8ee36f03-2c3b-4f1d-b2a5-aad0c2c351ab None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:43.346072 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-8ee36f03-2c3b-4f1d-b2a5-aad0c2c351ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:43.424105 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030fd5df-087a-4ae6-b3e2-2b3d5004697f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:43.425168 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030fd5df-087a-4ae6-b3e2-2b3d5004697f admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:43.425686 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030fd5df-087a-4ae6-b3e2-2b3d5004697f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:43.426200 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030fd5df-087a-4ae6-b3e2-2b3d5004697f admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:43.427103 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030fd5df-087a-4ae6-b3e2-2b3d5004697f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:43.429258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030fd5df-087a-4ae6-b3e2-2b3d5004697f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:43.479527 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 925/1850] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:43 2022] POST /identity/v3/users => generated 292 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:43.481294 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030fd5df-087a-4ae6-b3e2-2b3d5004697f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:43.487217 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-030fd5df-087a-4ae6-b3e2-2b3d5004697f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:43.509794 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030fd5df-087a-4ae6-b3e2-2b3d5004697f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:43.531000 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b09a6f-3609-49a8-aa18-bb8b43efeea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:43.532140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b09a6f-3609-49a8-aa18-bb8b43efeea5 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:43.532640 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b09a6f-3609-49a8-aa18-bb8b43efeea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:43.533182 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b09a6f-3609-49a8-aa18-bb8b43efeea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:43.534177 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b09a6f-3609-49a8-aa18-bb8b43efeea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:43.535983 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b09a6f-3609-49a8-aa18-bb8b43efeea5 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:43.548003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 926/1851] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:43.549204 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b09a6f-3609-49a8-aa18-bb8b43efeea5 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:43.560805 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b09a6f-3609-49a8-aa18-bb8b43efeea5 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:43.569049 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c1cd0-68bd-4ff6-a7f0-a0436875f7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:43.570142 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c1cd0-68bd-4ff6-a7f0-a0436875f7b7 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:43.570958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c1cd0-68bd-4ff6-a7f0-a0436875f7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:43.571305 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c1cd0-68bd-4ff6-a7f0-a0436875f7b7 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:43.572424 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c1cd0-68bd-4ff6-a7f0-a0436875f7b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:43.574213 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 926/1852] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:43.575170 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c1cd0-68bd-4ff6-a7f0-a0436875f7b7 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:43.576249 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee36f03-2c3b-4f1d-b2a5-aad0c2c351ab None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:43.596003 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee36f03-2c3b-4f1d-b2a5-aad0c2c351ab None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:43.602792 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031f56b-baff-4a07-bae1-77f6932a4ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:43.603991 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031f56b-baff-4a07-bae1-77f6932a4ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:43.604522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031f56b-baff-4a07-bae1-77f6932a4ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:43.605023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031f56b-baff-4a07-bae1-77f6932a4ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:43.606188 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5031f56b-baff-4a07-bae1-77f6932a4ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:43.610283 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 927/1853] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:43 2022] POST /identity/v3/users => generated 295 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:43.610283 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c1cd0-68bd-4ff6-a7f0-a0436875f7b7 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:43.612688 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5031f56b-baff-4a07-bae1-77f6932a4ef3 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:43.629407 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c1cd0-68bd-4ff6-a7f0-a0436875f7b7 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:43.630061 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 927/1854] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:43.633027 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af1192-2b42-4ea5-b665-c63024422979 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:43.633647 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af1192-2b42-4ea5-b665-c63024422979 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:43.634071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af1192-2b42-4ea5-b665-c63024422979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:43.640124 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bae9a0-873e-44ce-83d1-74c11f168f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:43.642267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bae9a0-873e-44ce-83d1-74c11f168f26 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:43.642942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bae9a0-873e-44ce-83d1-74c11f168f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:43.643402 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bae9a0-873e-44ce-83d1-74c11f168f26 admin admin] PATH_INFO: `/v3/projects/dc181a40b42a4f51982126b1bcbfebb2/users/94e7ebc9886c44d387f701e2a3e1dab3/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:43.647327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bae9a0-873e-44ce-83d1-74c11f168f26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc181a40b42a4f51982126b1bcbfebb2, user_id=94e7ebc9886c44d387f701e2a3e1dab3, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:43.650530 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bae9a0-873e-44ce-83d1-74c11f168f26 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:43.662710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 928/1855] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:43 2022] PUT /identity/v3/projects/dc181a40b42a4f51982126b1bcbfebb2/users/94e7ebc9886c44d387f701e2a3e1dab3/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:43.663582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bae9a0-873e-44ce-83d1-74c11f168f26 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:43.665162 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f2bae9a0-873e-44ce-83d1-74c11f168f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:43.680511 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-05af1192-2b42-4ea5-b665-c63024422979 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:43.681401 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-05af1192-2b42-4ea5-b665-c63024422979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:43.750851 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bae9a0-873e-44ce-83d1-74c11f168f26 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:43.755432 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d62dae-3e10-483e-b5cc-ce3e4c8aeeb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:43.756070 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d62dae-3e10-483e-b5cc-ce3e4c8aeeb9 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:43.756473 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d62dae-3e10-483e-b5cc-ce3e4c8aeeb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:43.756628 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d62dae-3e10-483e-b5cc-ce3e4c8aeeb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:43.758445 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a2d62dae-3e10-483e-b5cc-ce3e4c8aeeb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:43.770422 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 928/1856] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:43.770422 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af1192-2b42-4ea5-b665-c63024422979 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:43.781662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af1192-2b42-4ea5-b665-c63024422979 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:43.787301 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc3ad5-6b0b-4b92-b655-9b6a6aff52ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:43.788019 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc3ad5-6b0b-4b92-b655-9b6a6aff52ce admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:43.788300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc3ad5-6b0b-4b92-b655-9b6a6aff52ce admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:43.788570 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc3ad5-6b0b-4b92-b655-9b6a6aff52ce admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:43.789371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc3ad5-6b0b-4b92-b655-9b6a6aff52ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:43.791520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc3ad5-6b0b-4b92-b655-9b6a6aff52ce admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:43.805857 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 929/1857] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:43.805857 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc3ad5-6b0b-4b92-b655-9b6a6aff52ce admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:43.817492 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc3ad5-6b0b-4b92-b655-9b6a6aff52ce admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:43.822193 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7cc77e-9e24-45ae-954d-db3cd9a2c963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:43.822446 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d62dae-3e10-483e-b5cc-ce3e4c8aeeb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:43.822851 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7cc77e-9e24-45ae-954d-db3cd9a2c963 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:43.823128 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7cc77e-9e24-45ae-954d-db3cd9a2c963 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:43.823402 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7cc77e-9e24-45ae-954d-db3cd9a2c963 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:43.823917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d62dae-3e10-483e-b5cc-ce3e4c8aeeb9 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:43.824103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7cc77e-9e24-45ae-954d-db3cd9a2c963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:43.825666 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7cc77e-9e24-45ae-954d-db3cd9a2c963 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:43.833506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 929/1858] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:43.836441 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 930/1859] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:43.837245 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7cc77e-9e24-45ae-954d-db3cd9a2c963 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:43.840258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ab36b-591a-40e5-a1f1-21bde099c315 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:43.840976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ab36b-591a-40e5-a1f1-21bde099c315 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:43.841476 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ab36b-591a-40e5-a1f1-21bde099c315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:43.848073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7cc77e-9e24-45ae-954d-db3cd9a2c963 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:43.853403 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac9fbf-1600-4dbb-a17c-4028f722ff41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:43.853818 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac9fbf-1600-4dbb-a17c-4028f722ff41 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:43.854095 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac9fbf-1600-4dbb-a17c-4028f722ff41 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:43.854409 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac9fbf-1600-4dbb-a17c-4028f722ff41 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:43.855127 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac9fbf-1600-4dbb-a17c-4028f722ff41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:02:43.855428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac9fbf-1600-4dbb-a17c-4028f722ff41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:43.857424 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac9fbf-1600-4dbb-a17c-4028f722ff41 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:43.870896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 931/1860] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:02:43.871716 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac9fbf-1600-4dbb-a17c-4028f722ff41 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:43.881676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac9fbf-1600-4dbb-a17c-4028f722ff41 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:43.883649 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-582ab36b-591a-40e5-a1f1-21bde099c315 None None] MFA Rules not processed for user `94e7ebc9886c44d387f701e2a3e1dab3`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:43.885188 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-582ab36b-591a-40e5-a1f1-21bde099c315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:43.887579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab45ae-6b04-48c6-bc14-c98368032bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:43.888629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab45ae-6b04-48c6-bc14-c98368032bd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:43.888986 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab45ae-6b04-48c6-bc14-c98368032bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:43.889442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab45ae-6b04-48c6-bc14-c98368032bd4 admin admin] PATH_INFO: `/v3/projects/b5dfe9f2f52e431fbe01f9d4ffaef46c/users/ba59f4de88df490386e1387cfbeffd5f/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:43.905647 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab45ae-6b04-48c6-bc14-c98368032bd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5dfe9f2f52e431fbe01f9d4ffaef46c, user_id=ba59f4de88df490386e1387cfbeffd5f, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:43.918072 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab45ae-6b04-48c6-bc14-c98368032bd4 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:43.933957 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 932/1861] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:43 2022] PUT /identity/v3/projects/b5dfe9f2f52e431fbe01f9d4ffaef46c/users/ba59f4de88df490386e1387cfbeffd5f/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:43.934682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab45ae-6b04-48c6-bc14-c98368032bd4 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:43.989902 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 930/1862] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:43 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:43.990973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ab36b-591a-40e5-a1f1-21bde099c315 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:44.006401 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab45ae-6b04-48c6-bc14-c98368032bd4 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:44.012632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91da80-7489-4178-a852-3a560b7405cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:44.013772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91da80-7489-4178-a852-3a560b7405cb admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:44.014058 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91da80-7489-4178-a852-3a560b7405cb admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:44.014504 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91da80-7489-4178-a852-3a560b7405cb admin admin] PATH_INFO: `/v3/projects/f914ba072a714559b820e9db6c9e9f44/users/5823adc83b074721b296d56416153a3d/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:44.022357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ab36b-591a-40e5-a1f1-21bde099c315 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:44.040667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d44893-d7b6-4273-951f-e6b37b1e2ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:44.041029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91da80-7489-4178-a852-3a560b7405cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f914ba072a714559b820e9db6c9e9f44, user_id=5823adc83b074721b296d56416153a3d, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:44.041420 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d44893-d7b6-4273-951f-e6b37b1e2ad9 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:44.042139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d44893-d7b6-4273-951f-e6b37b1e2ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:44.042139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d44893-d7b6-4273-951f-e6b37b1e2ad9 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:44.042687 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d44893-d7b6-4273-951f-e6b37b1e2ad9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:44.044962 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d44893-d7b6-4273-951f-e6b37b1e2ad9 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:44.045081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91da80-7489-4178-a852-3a560b7405cb admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:44.076075 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 931/1863] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:43 2022] POST /identity/v3/projects => generated 383 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:44.076991 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d44893-d7b6-4273-951f-e6b37b1e2ad9 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:44.087278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 933/1864] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:43 2022] PUT /identity/v3/projects/f914ba072a714559b820e9db6c9e9f44/users/5823adc83b074721b296d56416153a3d/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:44.087957 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91da80-7489-4178-a852-3a560b7405cb admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:44.089693 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-fa91da80-7489-4178-a852-3a560b7405cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:44.163005 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91da80-7489-4178-a852-3a560b7405cb admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:44.172568 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d44893-d7b6-4273-951f-e6b37b1e2ad9 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:44.179931 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0c9ad-2f23-4eea-a3b7-1ac79c3106a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:44.181070 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0c9ad-2f23-4eea-a3b7-1ac79c3106a5 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:44.181633 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0c9ad-2f23-4eea-a3b7-1ac79c3106a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:44.182176 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0c9ad-2f23-4eea-a3b7-1ac79c3106a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:44.183840 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af0c9ad-2f23-4eea-a3b7-1ac79c3106a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:44.185746 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af0c9ad-2f23-4eea-a3b7-1ac79c3106a5 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:44.197450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 932/1865] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:44.198669 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0c9ad-2f23-4eea-a3b7-1ac79c3106a5 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:44.212983 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0c9ad-2f23-4eea-a3b7-1ac79c3106a5 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:44.220091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce0cbe2-241b-4187-8f84-7cdbbe37b056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:44.221651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0cbe2-241b-4187-8f84-7cdbbe37b056 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:44.222304 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0cbe2-241b-4187-8f84-7cdbbe37b056 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:44.222900 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0cbe2-241b-4187-8f84-7cdbbe37b056 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:44.223953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce0cbe2-241b-4187-8f84-7cdbbe37b056 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:44.226618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce0cbe2-241b-4187-8f84-7cdbbe37b056 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:44.228557 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e534d-917d-4834-bf8d-7caf3075baf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:44.228867 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e534d-917d-4834-bf8d-7caf3075baf3 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:44.229153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e534d-917d-4834-bf8d-7caf3075baf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:44.229427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e534d-917d-4834-bf8d-7caf3075baf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:44.232837 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-328e534d-917d-4834-bf8d-7caf3075baf3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:44.268218 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 933/1866] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:44 2022] POST /identity/v3/users => generated 292 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:44.269597 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc2803-2b79-44b1-aac8-cac3297632e3 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:44.269952 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc2803-2b79-44b1-aac8-cac3297632e3 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:44.270314 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc2803-2b79-44b1-aac8-cac3297632e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:44.308364 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-aafc2803-2b79-44b1-aac8-cac3297632e3 None None] MFA Rules not processed for user `5823adc83b074721b296d56416153a3d`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:44.309739 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-aafc2803-2b79-44b1-aac8-cac3297632e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:44.378250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e534d-917d-4834-bf8d-7caf3075baf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:44.388751 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e534d-917d-4834-bf8d-7caf3075baf3 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:44.405445 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 934/1867] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:44.407511 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e534d-917d-4834-bf8d-7caf3075baf3 service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:44.424606 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e534d-917d-4834-bf8d-7caf3075baf3 service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:44.431924 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c91b9-6f36-4e95-a3b5-8d9bd427614b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:44.432153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c91b9-6f36-4e95-a3b5-8d9bd427614b admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:44.432549 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c91b9-6f36-4e95-a3b5-8d9bd427614b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:44.432924 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c91b9-6f36-4e95-a3b5-8d9bd427614b admin admin] PATH_INFO: `/v3/projects/b5dfe9f2f52e431fbe01f9d4ffaef46c/users/ba59f4de88df490386e1387cfbeffd5f/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:44.435582 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c91b9-6f36-4e95-a3b5-8d9bd427614b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5dfe9f2f52e431fbe01f9d4ffaef46c, user_id=ba59f4de88df490386e1387cfbeffd5f, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:44.439054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c91b9-6f36-4e95-a3b5-8d9bd427614b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:44.456179 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 934/1868] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:44.459049 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc2803-2b79-44b1-aac8-cac3297632e3 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:44.467006 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 935/1869] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:44 2022] PUT /identity/v3/projects/b5dfe9f2f52e431fbe01f9d4ffaef46c/users/ba59f4de88df490386e1387cfbeffd5f/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:44.476257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8544659-6357-4c8f-8621-8cba8581b13f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:44.476978 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8544659-6357-4c8f-8621-8cba8581b13f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:44.477464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8544659-6357-4c8f-8621-8cba8581b13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:44.488171 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc2803-2b79-44b1-aac8-cac3297632e3 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:44.535124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c8544659-6357-4c8f-8621-8cba8581b13f None None] MFA Rules not processed for user `ba59f4de88df490386e1387cfbeffd5f`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:44.536530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c8544659-6357-4c8f-8621-8cba8581b13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:44.627136 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb547e-f295-4056-aaeb-ae36a2092920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:44.628597 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb547e-f295-4056-aaeb-ae36a2092920 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:44.628888 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb547e-f295-4056-aaeb-ae36a2092920 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:44.629528 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb547e-f295-4056-aaeb-ae36a2092920 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:44.632573 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abb547e-f295-4056-aaeb-ae36a2092920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:44.639139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abb547e-f295-4056-aaeb-ae36a2092920 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:44.659450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 935/1870] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:44.660144 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb547e-f295-4056-aaeb-ae36a2092920 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:44.719408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 936/1871] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:44 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:44.721887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8544659-6357-4c8f-8621-8cba8581b13f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:44.740122 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb547e-f295-4056-aaeb-ae36a2092920 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:44.744085 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727a030-b431-42a9-9b1a-373e1d0a63de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:44.744320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8544659-6357-4c8f-8621-8cba8581b13f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:44.745103 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727a030-b431-42a9-9b1a-373e1d0a63de service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:44.745616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727a030-b431-42a9-9b1a-373e1d0a63de service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:44.746058 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727a030-b431-42a9-9b1a-373e1d0a63de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:44.750647 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d12139-41b3-44fb-89d9-683798eb5640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:44.751396 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d12139-41b3-44fb-89d9-683798eb5640 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:44.751739 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d12139-41b3-44fb-89d9-683798eb5640 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:44.752071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d12139-41b3-44fb-89d9-683798eb5640 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:44.752938 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d12139-41b3-44fb-89d9-683798eb5640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:44.754771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d12139-41b3-44fb-89d9-683798eb5640 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:44.768170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 937/1872] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:44.772887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d12139-41b3-44fb-89d9-683798eb5640 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:44.787250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d12139-41b3-44fb-89d9-683798eb5640 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:44.797152 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486313a-9a88-4aaa-bee8-5f48fb8589b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:44.798311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486313a-9a88-4aaa-bee8-5f48fb8589b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:44.798597 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486313a-9a88-4aaa-bee8-5f48fb8589b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:44.798888 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486313a-9a88-4aaa-bee8-5f48fb8589b4 admin admin] PATH_INFO: `/v3/projects/f4ef87c3541c4c8e857754250a344557/users/072a5be9527a4dde8c64e5c25c06568c/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:44.816935 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f486313a-9a88-4aaa-bee8-5f48fb8589b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4ef87c3541c4c8e857754250a344557, user_id=072a5be9527a4dde8c64e5c25c06568c, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:44.819913 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f486313a-9a88-4aaa-bee8-5f48fb8589b4 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:44.836149 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 938/1873] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:44 2022] PUT /identity/v3/projects/f4ef87c3541c4c8e857754250a344557/users/072a5be9527a4dde8c64e5c25c06568c/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:44.837936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b727a030-b431-42a9-9b1a-373e1d0a63de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:44.839630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b727a030-b431-42a9-9b1a-373e1d0a63de service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:44.853099 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 936/1874] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:44.856623 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9d38d-90ae-4bb2-bad3-14ef581305c4 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:44.856982 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9d38d-90ae-4bb2-bad3-14ef581305c4 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:44.857587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9d38d-90ae-4bb2-bad3-14ef581305c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:44.886608 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-19f9d38d-90ae-4bb2-bad3-14ef581305c4 None None] MFA Rules not processed for user `072a5be9527a4dde8c64e5c25c06568c`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:44.887886 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-19f9d38d-90ae-4bb2-bad3-14ef581305c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:44.905329 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727a030-b431-42a9-9b1a-373e1d0a63de service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:44.913969 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727a030-b431-42a9-9b1a-373e1d0a63de service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:44.968818 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 939/1875] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:44.973192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9d38d-90ae-4bb2-bad3-14ef581305c4 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:45.016259 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c68837-e142-42ba-a27b-63b2a5e21fd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:45.016931 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c68837-e142-42ba-a27b-63b2a5e21fd0 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:45.018172 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c68837-e142-42ba-a27b-63b2a5e21fd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:45.018427 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c68837-e142-42ba-a27b-63b2a5e21fd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:45.049552 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9d38d-90ae-4bb2-bad3-14ef581305c4 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:45.054751 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8e6ca-4462-4446-aefa-484eac963ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:45.055515 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8e6ca-4462-4446-aefa-484eac963ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:45.056234 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8e6ca-4462-4446-aefa-484eac963ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:45.056582 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8e6ca-4462-4446-aefa-484eac963ef7 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:45.057660 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea8e6ca-4462-4446-aefa-484eac963ef7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:45.059353 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea8e6ca-4462-4446-aefa-484eac963ef7 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:45.108880 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 940/1876] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:44 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:02:45.110109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8e6ca-4462-4446-aefa-484eac963ef7 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:45.122551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c68837-e142-42ba-a27b-63b2a5e21fd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:45.124000 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c68837-e142-42ba-a27b-63b2a5e21fd0 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:45.148354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 937/1877] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:45.189124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8e6ca-4462-4446-aefa-484eac963ef7 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:45.204429 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ede58-fb19-4d1c-b99f-f5ce223c3e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:45.205443 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ede58-fb19-4d1c-b99f-f5ce223c3e4d admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:45.205823 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ede58-fb19-4d1c-b99f-f5ce223c3e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:45.206271 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ede58-fb19-4d1c-b99f-f5ce223c3e4d admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:45.207761 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ede58-fb19-4d1c-b99f-f5ce223c3e4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:45.210173 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ede58-fb19-4d1c-b99f-f5ce223c3e4d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:45.246045 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c68837-e142-42ba-a27b-63b2a5e21fd0 service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:45.254526 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 941/1878] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:45 2022] POST /identity/v3/users => generated 298 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:02:45.257531 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ede58-fb19-4d1c-b99f-f5ce223c3e4d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:45.259594 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c68837-e142-42ba-a27b-63b2a5e21fd0 service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:45.274995 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ede58-fb19-4d1c-b99f-f5ce223c3e4d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:45.282351 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac78940-2ec5-4cee-89d1-a37ee8cbaf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:45.283272 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac78940-2ec5-4cee-89d1-a37ee8cbaf77 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:45.283666 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac78940-2ec5-4cee-89d1-a37ee8cbaf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:45.283819 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac78940-2ec5-4cee-89d1-a37ee8cbaf77 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:45.287146 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac78940-2ec5-4cee-89d1-a37ee8cbaf77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:45.287146 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac78940-2ec5-4cee-89d1-a37ee8cbaf77 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:45.303414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 942/1879] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:45.304309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac78940-2ec5-4cee-89d1-a37ee8cbaf77 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:45.315406 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac78940-2ec5-4cee-89d1-a37ee8cbaf77 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:45.320619 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe8f80-716d-4e81-9ae3-10200321baba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:45.321612 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe8f80-716d-4e81-9ae3-10200321baba admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:45.321896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe8f80-716d-4e81-9ae3-10200321baba admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:45.322213 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe8f80-716d-4e81-9ae3-10200321baba admin admin] PATH_INFO: `/v3/projects/266aca812e954458b119d02d32af7b9b/users/b8808335bb244a689402db8f735cabd1/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:45.345397 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe8f80-716d-4e81-9ae3-10200321baba admin admin] RBAC: Authorizing `identity:create_grant(project_id=266aca812e954458b119d02d32af7b9b, user_id=b8808335bb244a689402db8f735cabd1, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:45.350388 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe8f80-716d-4e81-9ae3-10200321baba admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:45.365769 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5f99b-dc46-42ef-b18e-e77799bb7c60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:45.367035 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5f99b-dc46-42ef-b18e-e77799bb7c60 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:45.367610 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5f99b-dc46-42ef-b18e-e77799bb7c60 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:45.368176 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5f99b-dc46-42ef-b18e-e77799bb7c60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:45.368969 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 943/1880] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:45 2022] PUT /identity/v3/projects/266aca812e954458b119d02d32af7b9b/users/b8808335bb244a689402db8f735cabd1/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:45.369981 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-29a5f99b-dc46-42ef-b18e-e77799bb7c60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:45.373261 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe8f80-716d-4e81-9ae3-10200321baba admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:45.448091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a5f99b-dc46-42ef-b18e-e77799bb7c60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:45.449060 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe8f80-716d-4e81-9ae3-10200321baba admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:45.449742 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a5f99b-dc46-42ef-b18e-e77799bb7c60 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:45.455991 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8c0b4-b167-4699-927f-876364edb53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:45.456700 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8c0b4-b167-4699-927f-876364edb53f admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:45.456984 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8c0b4-b167-4699-927f-876364edb53f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:45.457304 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8c0b4-b167-4699-927f-876364edb53f admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:45.457977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d8c0b4-b167-4699-927f-876364edb53f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:45.459451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 938/1881] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 23:02:45.460086 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d8c0b4-b167-4699-927f-876364edb53f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:45.472385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 944/1882] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:02:45.476829 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5f99b-dc46-42ef-b18e-e77799bb7c60 service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:45.495226 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5f99b-dc46-42ef-b18e-e77799bb7c60 service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:45.500448 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae1566-b74a-4a2b-b367-2fe3cc68b21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:45.501431 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae1566-b74a-4a2b-b367-2fe3cc68b21f admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:45.501929 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae1566-b74a-4a2b-b367-2fe3cc68b21f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:45.502321 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae1566-b74a-4a2b-b367-2fe3cc68b21f admin admin] PATH_INFO: `/v3/projects/266aca812e954458b119d02d32af7b9b/users/b8808335bb244a689402db8f735cabd1/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:45.504429 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae1566-b74a-4a2b-b367-2fe3cc68b21f admin admin] RBAC: Authorizing `identity:create_grant(project_id=266aca812e954458b119d02d32af7b9b, user_id=b8808335bb244a689402db8f735cabd1, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:45.507555 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae1566-b74a-4a2b-b367-2fe3cc68b21f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:45.520348 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 939/1883] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:45 2022] PUT /identity/v3/projects/266aca812e954458b119d02d32af7b9b/users/b8808335bb244a689402db8f735cabd1/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:45.532879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66992e62-6bda-4c6d-b113-d203753d99d8 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:45.533164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66992e62-6bda-4c6d-b113-d203753d99d8 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:45.533439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66992e62-6bda-4c6d-b113-d203753d99d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:45.569458 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-66992e62-6bda-4c6d-b113-d203753d99d8 None None] MFA Rules not processed for user `b8808335bb244a689402db8f735cabd1`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:45.571482 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-66992e62-6bda-4c6d-b113-d203753d99d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:45.700220 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 945/1884] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:45.737884 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae1566-b74a-4a2b-b367-2fe3cc68b21f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:45.805747 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae1566-b74a-4a2b-b367-2fe3cc68b21f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:45.809985 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58602e78-69a7-4d02-8de2-3f7e3c24c8a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:45.810720 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58602e78-69a7-4d02-8de2-3f7e3c24c8a5 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:45.811065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58602e78-69a7-4d02-8de2-3f7e3c24c8a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:45.811519 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58602e78-69a7-4d02-8de2-3f7e3c24c8a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:45.824655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58602e78-69a7-4d02-8de2-3f7e3c24c8a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:45.826206 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58602e78-69a7-4d02-8de2-3f7e3c24c8a5 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:45.846461 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 940/1885] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:46.009070 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66992e62-6bda-4c6d-b113-d203753d99d8 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:46.024946 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66992e62-6bda-4c6d-b113-d203753d99d8 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:46.029213 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8cf64-7b97-432a-95a8-76be14cce885 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:46.029967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8cf64-7b97-432a-95a8-76be14cce885 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:46.030634 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8cf64-7b97-432a-95a8-76be14cce885 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:46.031108 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8cf64-7b97-432a-95a8-76be14cce885 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:46.100474 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8cf64-7b97-432a-95a8-76be14cce885 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:46.102483 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8cf64-7b97-432a-95a8-76be14cce885 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:46.118410 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 946/1886] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:48.825420 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58602e78-69a7-4d02-8de2-3f7e3c24c8a5 service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:48.833984 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58602e78-69a7-4d02-8de2-3f7e3c24c8a5 service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:48.837806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5594770-1b30-4c4b-ac01-0b7efde4f522 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:48.838538 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5594770-1b30-4c4b-ac01-0b7efde4f522 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:48.838882 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5594770-1b30-4c4b-ac01-0b7efde4f522 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:48.839221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5594770-1b30-4c4b-ac01-0b7efde4f522 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:48.900208 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5594770-1b30-4c4b-ac01-0b7efde4f522 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:48.901720 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5594770-1b30-4c4b-ac01-0b7efde4f522 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:48.911414 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 941/1887] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:48 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:55.307748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55308af-0397-4d92-90e1-26f9ef2b7ef5 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:55.308737 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55308af-0397-4d92-90e1-26f9ef2b7ef5 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:55.310034 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55308af-0397-4d92-90e1-26f9ef2b7ef5 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:55.312807 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 947/1888] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 9 23:02:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 23:02:55.331193 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f964492-4099-4058-a769-e0d5731fd3f5 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:55.331774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f964492-4099-4058-a769-e0d5731fd3f5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:55.332163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f964492-4099-4058-a769-e0d5731fd3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:55.383025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-4f964492-4099-4058-a769-e0d5731fd3f5 None None] MFA Rules not processed for user `02fcd7cc86134ce488c7dd75dfb174b8`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:55.385195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-4f964492-4099-4058-a769-e0d5731fd3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:55.515470 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 942/1889] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 9 23:02:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:55.536689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55308af-0397-4d92-90e1-26f9ef2b7ef5 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:55.541502 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c55308af-0397-4d92-90e1-26f9ef2b7ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:55.582625 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55308af-0397-4d92-90e1-26f9ef2b7ef5 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:55.596547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d53a84-bfbb-4174-8a33-44a03dbbb512 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:55.597703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d53a84-bfbb-4174-8a33-44a03dbbb512 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:55.598400 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d53a84-bfbb-4174-8a33-44a03dbbb512 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:55.598858 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d53a84-bfbb-4174-8a33-44a03dbbb512 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:55.619503 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d53a84-bfbb-4174-8a33-44a03dbbb512 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:55.627303 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d53a84-bfbb-4174-8a33-44a03dbbb512 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:55.651287 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 948/1890] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:55.973567 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f964492-4099-4058-a769-e0d5731fd3f5 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:56.081901 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f964492-4099-4058-a769-e0d5731fd3f5 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:56.089816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d53a84-bfbb-4174-8a33-44a03dbbb512 service nova] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:56.089965 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d4df2-a4dd-4596-9227-ee56b452c317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:56.091468 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d4df2-a4dd-4596-9227-ee56b452c317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:56.093394 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d4df2-a4dd-4596-9227-ee56b452c317 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:56.094089 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d4df2-a4dd-4596-9227-ee56b452c317 admin admin] PATH_INFO: `/v3/users/4b51b0be36c24568aab90516e3c29916` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:56.099055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d4df2-a4dd-4596-9227-ee56b452c317 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b51b0be36c24568aab90516e3c29916)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:56.102957 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d4df2-a4dd-4596-9227-ee56b452c317 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:56.106409 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d53a84-bfbb-4174-8a33-44a03dbbb512 service nova] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:56.113285 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a997b17-c9fc-49b3-859f-95b37f5c4960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:56.115933 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a997b17-c9fc-49b3-859f-95b37f5c4960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:56.116218 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a997b17-c9fc-49b3-859f-95b37f5c4960 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:56.116556 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a997b17-c9fc-49b3-859f-95b37f5c4960 admin admin] PATH_INFO: `/v3/users/072a5be9527a4dde8c64e5c25c06568c` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:56.118324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a997b17-c9fc-49b3-859f-95b37f5c4960 admin admin] RBAC: Authorizing `identity:delete_user(user_id=072a5be9527a4dde8c64e5c25c06568c)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:56.120497 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a997b17-c9fc-49b3-859f-95b37f5c4960 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:56.178064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-104d4df2-a4dd-4596-9227-ee56b452c317 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b51b0be36c24568aab90516e3c29916'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:56.188235 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-104d4df2-a4dd-4596-9227-ee56b452c317 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b51b0be36c24568aab90516e3c29916'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:56.190199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-104d4df2-a4dd-4596-9227-ee56b452c317 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b51b0be36c24568aab90516e3c29916'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:56.200771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-104d4df2-a4dd-4596-9227-ee56b452c317 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b51b0be36c24568aab90516e3c29916'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:56.201771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-104d4df2-a4dd-4596-9227-ee56b452c317 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b51b0be36c24568aab90516e3c29916'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:56.205283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8a997b17-c9fc-49b3-859f-95b37f5c4960 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '072a5be9527a4dde8c64e5c25c06568c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:56.215649 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8a997b17-c9fc-49b3-859f-95b37f5c4960 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '072a5be9527a4dde8c64e5c25c06568c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:56.227512 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8a997b17-c9fc-49b3-859f-95b37f5c4960 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '072a5be9527a4dde8c64e5c25c06568c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:56.228912 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8a997b17-c9fc-49b3-859f-95b37f5c4960 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '072a5be9527a4dde8c64e5c25c06568c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:56.230212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 943/1891] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:55 2022] DELETE /identity/v3/users/4b51b0be36c24568aab90516e3c29916 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:56.231398 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d4df2-a4dd-4596-9227-ee56b452c317 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:56.233328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-104d4df2-a4dd-4596-9227-ee56b452c317 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:56.247582 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8a997b17-c9fc-49b3-859f-95b37f5c4960 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '072a5be9527a4dde8c64e5c25c06568c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:56.269223 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 949/1892] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:56 2022] DELETE /identity/v3/users/072a5be9527a4dde8c64e5c25c06568c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:56.271094 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f3f45-8f43-4e08-b9ed-483cbcf9a7f5 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:56.271524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f3f45-8f43-4e08-b9ed-483cbcf9a7f5 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:56.271948 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f3f45-8f43-4e08-b9ed-483cbcf9a7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:56.289923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-ce3f3f45-8f43-4e08-b9ed-483cbcf9a7f5 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:56.291372 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ce3f3f45-8f43-4e08-b9ed-483cbcf9a7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:56.309349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d4df2-a4dd-4596-9227-ee56b452c317 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:56.361311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 950/1893] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:56.362107 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc012bd-46a6-4ebf-a0dd-39703f693b8f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:56.362455 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc012bd-46a6-4ebf-a0dd-39703f693b8f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:56.362721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc012bd-46a6-4ebf-a0dd-39703f693b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:56.379151 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c620f97-9a03-4856-9390-d1fab9730434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:56.380212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c620f97-9a03-4856-9390-d1fab9730434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:56.380494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c620f97-9a03-4856-9390-d1fab9730434 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:56.381330 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c620f97-9a03-4856-9390-d1fab9730434 admin admin] PATH_INFO: `/v3/users/dd979890cd2040ee8779b8a6a7bdcc41` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:56.382199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c620f97-9a03-4856-9390-d1fab9730434 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd979890cd2040ee8779b8a6a7bdcc41)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:56.384119 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c620f97-9a03-4856-9390-d1fab9730434 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:56.388181 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-0fc012bd-46a6-4ebf-a0dd-39703f693b8f None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:56.389427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0fc012bd-46a6-4ebf-a0dd-39703f693b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:56.397362 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 951/1894] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:56.398193 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc012bd-46a6-4ebf-a0dd-39703f693b8f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:56.399428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0fc012bd-46a6-4ebf-a0dd-39703f693b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:56.433698 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-0c620f97-9a03-4856-9390-d1fab9730434 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd979890cd2040ee8779b8a6a7bdcc41'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:56.440966 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-0c620f97-9a03-4856-9390-d1fab9730434 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd979890cd2040ee8779b8a6a7bdcc41'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:56.441651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-0c620f97-9a03-4856-9390-d1fab9730434 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd979890cd2040ee8779b8a6a7bdcc41'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:56.450799 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-0c620f97-9a03-4856-9390-d1fab9730434 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd979890cd2040ee8779b8a6a7bdcc41'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:56.451436 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-0c620f97-9a03-4856-9390-d1fab9730434 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd979890cd2040ee8779b8a6a7bdcc41'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:56.460936 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc012bd-46a6-4ebf-a0dd-39703f693b8f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:56.468048 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358caf5-f54c-4af6-9f7f-0dca75a7bcbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:56.468772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358caf5-f54c-4af6-9f7f-0dca75a7bcbd service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:56.469056 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358caf5-f54c-4af6-9f7f-0dca75a7bcbd service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:56.469872 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358caf5-f54c-4af6-9f7f-0dca75a7bcbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:56.471789 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5358caf5-f54c-4af6-9f7f-0dca75a7bcbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:56.473767 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 944/1895] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:56 2022] DELETE /identity/v3/users/dd979890cd2040ee8779b8a6a7bdcc41 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:56.474632 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c620f97-9a03-4856-9390-d1fab9730434 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:56.475966 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0c620f97-9a03-4856-9390-d1fab9730434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:56.487073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5358caf5-f54c-4af6-9f7f-0dca75a7bcbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:56.488602 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5358caf5-f54c-4af6-9f7f-0dca75a7bcbd service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:56.499187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 952/1896] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:56.500615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4620948-b6be-4807-ba33-2a028f949581 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:56.500905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4620948-b6be-4807-ba33-2a028f949581 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:56.501187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4620948-b6be-4807-ba33-2a028f949581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:56.522085 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e4620948-b6be-4807-ba33-2a028f949581 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:56.524258 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e4620948-b6be-4807-ba33-2a028f949581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:56.551804 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c620f97-9a03-4856-9390-d1fab9730434 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:56.559461 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33bd29-4bb5-49a4-a320-c45e5006e1a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:56.560928 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33bd29-4bb5-49a4-a320-c45e5006e1a0 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:56.562484 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33bd29-4bb5-49a4-a320-c45e5006e1a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:56.563065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33bd29-4bb5-49a4-a320-c45e5006e1a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:56.567014 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ff33bd29-4bb5-49a4-a320-c45e5006e1a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:56.596185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 953/1897] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:56.604320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4620948-b6be-4807-ba33-2a028f949581 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:56.605754 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e4620948-b6be-4807-ba33-2a028f949581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:56.620015 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4620948-b6be-4807-ba33-2a028f949581 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:56.632090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f746f2c-30a5-401a-976f-59d7abc77d5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:56.633255 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f746f2c-30a5-401a-976f-59d7abc77d5b service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:56.633750 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f746f2c-30a5-401a-976f-59d7abc77d5b service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:56.634693 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f746f2c-30a5-401a-976f-59d7abc77d5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:56.646377 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f746f2c-30a5-401a-976f-59d7abc77d5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:56.647976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f746f2c-30a5-401a-976f-59d7abc77d5b service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:56.658824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 954/1898] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:56.665245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff33bd29-4bb5-49a4-a320-c45e5006e1a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:56.666746 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff33bd29-4bb5-49a4-a320-c45e5006e1a0 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:56.681244 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 945/1899] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:56.879971 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f746f2c-30a5-401a-976f-59d7abc77d5b service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:56.882194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-8f746f2c-30a5-401a-976f-59d7abc77d5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:56.896421 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f746f2c-30a5-401a-976f-59d7abc77d5b service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:56.902187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2841b-6eab-451f-b441-c24011e508db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:56.903492 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2841b-6eab-451f-b441-c24011e508db admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:56.903841 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2841b-6eab-451f-b441-c24011e508db admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:56.904146 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2841b-6eab-451f-b441-c24011e508db admin admin] PATH_INFO: `/v3/users/5823adc83b074721b296d56416153a3d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:56.905941 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2841b-6eab-451f-b441-c24011e508db admin admin] RBAC: Authorizing `identity:delete_user(user_id=5823adc83b074721b296d56416153a3d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:56.908799 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2841b-6eab-451f-b441-c24011e508db admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:56.944522 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33bd29-4bb5-49a4-a320-c45e5006e1a0 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:56.945945 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ff33bd29-4bb5-49a4-a320-c45e5006e1a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:56.966261 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33bd29-4bb5-49a4-a320-c45e5006e1a0 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:56.972917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530a9e5-2c44-49de-9bbc-d6299364b983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:56.973894 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530a9e5-2c44-49de-9bbc-d6299364b983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:56.974194 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530a9e5-2c44-49de-9bbc-d6299364b983 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:56.975235 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530a9e5-2c44-49de-9bbc-d6299364b983 admin admin] PATH_INFO: `/v3/projects/6ae66866abe84d6596765919b1f546dd` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:56.976409 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530a9e5-2c44-49de-9bbc-d6299364b983 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ae66866abe84d6596765919b1f546dd)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:56.978788 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530a9e5-2c44-49de-9bbc-d6299364b983 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:57.020644 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9dc2841b-6eab-451f-b441-c24011e508db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5823adc83b074721b296d56416153a3d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.039046 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9dc2841b-6eab-451f-b441-c24011e508db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5823adc83b074721b296d56416153a3d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.053849 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9dc2841b-6eab-451f-b441-c24011e508db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5823adc83b074721b296d56416153a3d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.054507 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9dc2841b-6eab-451f-b441-c24011e508db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5823adc83b074721b296d56416153a3d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.074875 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9dc2841b-6eab-451f-b441-c24011e508db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5823adc83b074721b296d56416153a3d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.086804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 955/1900] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:56 2022] DELETE /identity/v3/users/5823adc83b074721b296d56416153a3d => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:57.089999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bf053-e315-442a-8ffd-a4258e8e4a0b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:57.090319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bf053-e315-442a-8ffd-a4258e8e4a0b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:57.090598 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bf053-e315-442a-8ffd-a4258e8e4a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:57.105137 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-9530a9e5-2c44-49de-9bbc-d6299364b983 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ae66866abe84d6596765919b1f546dd', 'request_id': 'req-9530a9e5-2c44-49de-9bbc-d6299364b983'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.117119 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-9530a9e5-2c44-49de-9bbc-d6299364b983 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ae66866abe84d6596765919b1f546dd', 'request_id': 'req-9530a9e5-2c44-49de-9bbc-d6299364b983'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.120658 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-ec8bf053-e315-442a-8ffd-a4258e8e4a0b None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:57.121892 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ec8bf053-e315-442a-8ffd-a4258e8e4a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:57.133620 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-9530a9e5-2c44-49de-9bbc-d6299364b983 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ae66866abe84d6596765919b1f546dd', 'request_id': 'req-9530a9e5-2c44-49de-9bbc-d6299364b983'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.159327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-9530a9e5-2c44-49de-9bbc-d6299364b983 admin admin] The token cache is being invalidate because project 6ae66866abe84d6596765919b1f546dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:57.159624 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-9530a9e5-2c44-49de-9bbc-d6299364b983 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.160990 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-9530a9e5-2c44-49de-9bbc-d6299364b983 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.168990 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 946/1901] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:56 2022] DELETE /identity/v3/projects/6ae66866abe84d6596765919b1f546dd => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:57.172869 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530a9e5-2c44-49de-9bbc-d6299364b983 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:57.184426 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9530a9e5-2c44-49de-9bbc-d6299364b983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:57.259121 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 956/1902] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:57.260866 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bf053-e315-442a-8ffd-a4258e8e4a0b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:57.265550 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ec8bf053-e315-442a-8ffd-a4258e8e4a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:57.300874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bf053-e315-442a-8ffd-a4258e8e4a0b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:57.306506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530a9e5-2c44-49de-9bbc-d6299364b983 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:57.307003 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7265ca-848c-42ef-a856-b44a6a4b9ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:57.308792 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7265ca-848c-42ef-a856-b44a6a4b9ca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:57.309245 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7265ca-848c-42ef-a856-b44a6a4b9ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:57.309606 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7265ca-848c-42ef-a856-b44a6a4b9ca2 admin admin] PATH_INFO: `/v3/projects/f0c37f9323b04350a1a12ffd9ff4acce` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:57.316381 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcd01c-4edf-420a-9536-a64c9d153ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:57.317404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcd01c-4edf-420a-9536-a64c9d153ae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:57.317758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcd01c-4edf-420a-9536-a64c9d153ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:57.318101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcd01c-4edf-420a-9536-a64c9d153ae0 admin admin] PATH_INFO: `/v3/users/cdf2ab2b9a734a31bc7e0e194c267dde` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:57.319477 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dcd01c-4edf-420a-9536-a64c9d153ae0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdf2ab2b9a734a31bc7e0e194c267dde)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:57.322645 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dcd01c-4edf-420a-9536-a64c9d153ae0 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:57.322946 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7265ca-848c-42ef-a856-b44a6a4b9ca2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0c37f9323b04350a1a12ffd9ff4acce)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:57.325849 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7265ca-848c-42ef-a856-b44a6a4b9ca2 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:57.385944 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a9dcd01c-4edf-420a-9536-a64c9d153ae0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdf2ab2b9a734a31bc7e0e194c267dde'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.393878 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a9dcd01c-4edf-420a-9536-a64c9d153ae0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdf2ab2b9a734a31bc7e0e194c267dde'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.396474 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a9dcd01c-4edf-420a-9536-a64c9d153ae0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdf2ab2b9a734a31bc7e0e194c267dde'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.405109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a9dcd01c-4edf-420a-9536-a64c9d153ae0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdf2ab2b9a734a31bc7e0e194c267dde'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.406513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a9dcd01c-4edf-420a-9536-a64c9d153ae0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdf2ab2b9a734a31bc7e0e194c267dde'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.407579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4a7265ca-848c-42ef-a856-b44a6a4b9ca2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0c37f9323b04350a1a12ffd9ff4acce', 'request_id': 'req-4a7265ca-848c-42ef-a856-b44a6a4b9ca2'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.415374 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4a7265ca-848c-42ef-a856-b44a6a4b9ca2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0c37f9323b04350a1a12ffd9ff4acce', 'request_id': 'req-4a7265ca-848c-42ef-a856-b44a6a4b9ca2'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.422581 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4a7265ca-848c-42ef-a856-b44a6a4b9ca2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0c37f9323b04350a1a12ffd9ff4acce', 'request_id': 'req-4a7265ca-848c-42ef-a856-b44a6a4b9ca2'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.436758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 947/1903] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:57 2022] DELETE /identity/v3/users/cdf2ab2b9a734a31bc7e0e194c267dde => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:02:57.436758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcd01c-4edf-420a-9536-a64c9d153ae0 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:57.438836 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a9dcd01c-4edf-420a-9536-a64c9d153ae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:57.439803 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4a7265ca-848c-42ef-a856-b44a6a4b9ca2 admin admin] The token cache is being invalidate because project f0c37f9323b04350a1a12ffd9ff4acce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:57.440092 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4a7265ca-848c-42ef-a856-b44a6a4b9ca2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.440757 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4a7265ca-848c-42ef-a856-b44a6a4b9ca2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.442342 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 957/1904] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:57 2022] DELETE /identity/v3/projects/f0c37f9323b04350a1a12ffd9ff4acce => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:57.443377 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7265ca-848c-42ef-a856-b44a6a4b9ca2 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:57.444857 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4a7265ca-848c-42ef-a856-b44a6a4b9ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:57.502423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcd01c-4edf-420a-9536-a64c9d153ae0 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:57.507610 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da3777-c323-40f7-b01e-31e0ca1159fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:57.508357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da3777-c323-40f7-b01e-31e0ca1159fe service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:57.508831 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da3777-c323-40f7-b01e-31e0ca1159fe service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:57.508831 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da3777-c323-40f7-b01e-31e0ca1159fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:57.510833 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-14da3777-c323-40f7-b01e-31e0ca1159fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:57.525279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7265ca-848c-42ef-a856-b44a6a4b9ca2 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:57.582080 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da3777-c323-40f7-b01e-31e0ca1159fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:57.583103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a8b93-dca3-464d-a7fb-8572693b53f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:57.583657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da3777-c323-40f7-b01e-31e0ca1159fe service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:57.583961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a8b93-dca3-464d-a7fb-8572693b53f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:57.584239 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a8b93-dca3-464d-a7fb-8572693b53f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:57.584511 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a8b93-dca3-464d-a7fb-8572693b53f7 admin admin] PATH_INFO: `/v3/projects/f4ef87c3541c4c8e857754250a344557` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:57.585691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a8b93-dca3-464d-a7fb-8572693b53f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4ef87c3541c4c8e857754250a344557)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:57.587628 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a8b93-dca3-464d-a7fb-8572693b53f7 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:57.595786 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 948/1905] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:02:57.597011 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da3777-c323-40f7-b01e-31e0ca1159fe service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:57.599193 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-14da3777-c323-40f7-b01e-31e0ca1159fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:57.622182 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da3777-c323-40f7-b01e-31e0ca1159fe service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:57.633088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa18b98-9bf0-47e6-923e-4f6da7ca2562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:57.633969 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa18b98-9bf0-47e6-923e-4f6da7ca2562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:57.634355 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa18b98-9bf0-47e6-923e-4f6da7ca2562 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:57.634731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa18b98-9bf0-47e6-923e-4f6da7ca2562 admin admin] PATH_INFO: `/v3/users/07c94732921e4dbe9ac16a49d2b722bc` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:57.638165 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa18b98-9bf0-47e6-923e-4f6da7ca2562 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07c94732921e4dbe9ac16a49d2b722bc)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:57.638165 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa18b98-9bf0-47e6-923e-4f6da7ca2562 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:57.652153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-984a8b93-dca3-464d-a7fb-8572693b53f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4ef87c3541c4c8e857754250a344557', 'request_id': 'req-984a8b93-dca3-464d-a7fb-8572693b53f7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.660575 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-984a8b93-dca3-464d-a7fb-8572693b53f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4ef87c3541c4c8e857754250a344557', 'request_id': 'req-984a8b93-dca3-464d-a7fb-8572693b53f7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.669318 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-984a8b93-dca3-464d-a7fb-8572693b53f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4ef87c3541c4c8e857754250a344557', 'request_id': 'req-984a8b93-dca3-464d-a7fb-8572693b53f7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.684061 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-984a8b93-dca3-464d-a7fb-8572693b53f7 admin admin] The token cache is being invalidate because project f4ef87c3541c4c8e857754250a344557 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:57.684348 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-984a8b93-dca3-464d-a7fb-8572693b53f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.685040 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-984a8b93-dca3-464d-a7fb-8572693b53f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.687029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 958/1906] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:57 2022] DELETE /identity/v3/projects/f4ef87c3541c4c8e857754250a344557 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:57.691347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a8b93-dca3-464d-a7fb-8572693b53f7 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:57.691982 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-daa18b98-9bf0-47e6-923e-4f6da7ca2562 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07c94732921e4dbe9ac16a49d2b722bc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.693193 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-984a8b93-dca3-464d-a7fb-8572693b53f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:57.699930 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-daa18b98-9bf0-47e6-923e-4f6da7ca2562 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07c94732921e4dbe9ac16a49d2b722bc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.700652 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-daa18b98-9bf0-47e6-923e-4f6da7ca2562 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07c94732921e4dbe9ac16a49d2b722bc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.709724 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-daa18b98-9bf0-47e6-923e-4f6da7ca2562 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07c94732921e4dbe9ac16a49d2b722bc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.711535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-daa18b98-9bf0-47e6-923e-4f6da7ca2562 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07c94732921e4dbe9ac16a49d2b722bc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.753243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 949/1907] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:57 2022] DELETE /identity/v3/users/07c94732921e4dbe9ac16a49d2b722bc => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:57.786991 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a8b93-dca3-464d-a7fb-8572693b53f7 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:57.851355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60daa369-7d69-4e5c-bc18-f9bb3fca1fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:57.853364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60daa369-7d69-4e5c-bc18-f9bb3fca1fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:57.853364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60daa369-7d69-4e5c-bc18-f9bb3fca1fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:57.853364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60daa369-7d69-4e5c-bc18-f9bb3fca1fe9 admin admin] PATH_INFO: `/v3/users/b8808335bb244a689402db8f735cabd1` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:57.862852 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60daa369-7d69-4e5c-bc18-f9bb3fca1fe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8808335bb244a689402db8f735cabd1)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:57.866631 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60daa369-7d69-4e5c-bc18-f9bb3fca1fe9 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:57.935203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa18b98-9bf0-47e6-923e-4f6da7ca2562 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:57.937127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-daa18b98-9bf0-47e6-923e-4f6da7ca2562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:57.950950 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa18b98-9bf0-47e6-923e-4f6da7ca2562 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:57.955511 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-60daa369-7d69-4e5c-bc18-f9bb3fca1fe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8808335bb244a689402db8f735cabd1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.962948 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8181627-352c-46ce-be43-411b3019822f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:57.964115 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8181627-352c-46ce-be43-411b3019822f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:57.964427 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8181627-352c-46ce-be43-411b3019822f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:57.964857 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8181627-352c-46ce-be43-411b3019822f admin admin] PATH_INFO: `/v3/projects/1f4da2a0d8554ce69aebe1f016dcfe45` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:57.966206 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8181627-352c-46ce-be43-411b3019822f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f4da2a0d8554ce69aebe1f016dcfe45)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:57.968611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8181627-352c-46ce-be43-411b3019822f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:57.981018 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-60daa369-7d69-4e5c-bc18-f9bb3fca1fe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8808335bb244a689402db8f735cabd1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.994536 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-60daa369-7d69-4e5c-bc18-f9bb3fca1fe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8808335bb244a689402db8f735cabd1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:57.996791 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-60daa369-7d69-4e5c-bc18-f9bb3fca1fe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8808335bb244a689402db8f735cabd1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.021033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-60daa369-7d69-4e5c-bc18-f9bb3fca1fe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8808335bb244a689402db8f735cabd1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.032612 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 959/1908] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:57 2022] DELETE /identity/v3/users/b8808335bb244a689402db8f735cabd1 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:02:58.071805 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-c8181627-352c-46ce-be43-411b3019822f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f4da2a0d8554ce69aebe1f016dcfe45', 'request_id': 'req-c8181627-352c-46ce-be43-411b3019822f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.078577 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-c8181627-352c-46ce-be43-411b3019822f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f4da2a0d8554ce69aebe1f016dcfe45', 'request_id': 'req-c8181627-352c-46ce-be43-411b3019822f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.085191 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-c8181627-352c-46ce-be43-411b3019822f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f4da2a0d8554ce69aebe1f016dcfe45', 'request_id': 'req-c8181627-352c-46ce-be43-411b3019822f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.101073 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-c8181627-352c-46ce-be43-411b3019822f admin admin] The token cache is being invalidate because project 1f4da2a0d8554ce69aebe1f016dcfe45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:58.101489 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-c8181627-352c-46ce-be43-411b3019822f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.112779 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-c8181627-352c-46ce-be43-411b3019822f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.117790 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 950/1909] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:57 2022] DELETE /identity/v3/projects/1f4da2a0d8554ce69aebe1f016dcfe45 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:02:58.121786 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60daa369-7d69-4e5c-bc18-f9bb3fca1fe9 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:58.125874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-60daa369-7d69-4e5c-bc18-f9bb3fca1fe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:58.210532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60daa369-7d69-4e5c-bc18-f9bb3fca1fe9 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:58.276947 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8181627-352c-46ce-be43-411b3019822f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:58.279396 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c8181627-352c-46ce-be43-411b3019822f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:58.287778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d2dc7-fcc8-4094-89e3-7a672b74a8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:58.288655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d2dc7-fcc8-4094-89e3-7a672b74a8fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:58.288932 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d2dc7-fcc8-4094-89e3-7a672b74a8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:58.289228 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d2dc7-fcc8-4094-89e3-7a672b74a8fe admin admin] PATH_INFO: `/v3/users/ba59f4de88df490386e1387cfbeffd5f` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:58.290450 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d2dc7-fcc8-4094-89e3-7a672b74a8fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba59f4de88df490386e1387cfbeffd5f)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:58.292366 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d2dc7-fcc8-4094-89e3-7a672b74a8fe admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:58.347318 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2f4d2dc7-fcc8-4094-89e3-7a672b74a8fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba59f4de88df490386e1387cfbeffd5f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.354590 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2f4d2dc7-fcc8-4094-89e3-7a672b74a8fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba59f4de88df490386e1387cfbeffd5f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.364136 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2f4d2dc7-fcc8-4094-89e3-7a672b74a8fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba59f4de88df490386e1387cfbeffd5f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.364886 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2f4d2dc7-fcc8-4094-89e3-7a672b74a8fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba59f4de88df490386e1387cfbeffd5f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.377379 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2f4d2dc7-fcc8-4094-89e3-7a672b74a8fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba59f4de88df490386e1387cfbeffd5f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.384495 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 960/1910] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:58 2022] DELETE /identity/v3/users/ba59f4de88df490386e1387cfbeffd5f => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:58.385559 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d2dc7-fcc8-4094-89e3-7a672b74a8fe admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:58.386184 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8181627-352c-46ce-be43-411b3019822f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:58.388665 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-2f4d2dc7-fcc8-4094-89e3-7a672b74a8fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:58.453308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e56888-6bf6-4551-a8a0-2020529998e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:58.454767 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e56888-6bf6-4551-a8a0-2020529998e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:58.455044 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e56888-6bf6-4551-a8a0-2020529998e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:58.455312 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e56888-6bf6-4551-a8a0-2020529998e0 admin admin] PATH_INFO: `/v3/projects/f914ba072a714559b820e9db6c9e9f44` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:58.456517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e56888-6bf6-4551-a8a0-2020529998e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f914ba072a714559b820e9db6c9e9f44)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:58.458938 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e56888-6bf6-4551-a8a0-2020529998e0 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:58.463464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d2dc7-fcc8-4094-89e3-7a672b74a8fe admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:58.469127 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509a452-b76c-4d85-b47d-60769a739666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:58.470648 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509a452-b76c-4d85-b47d-60769a739666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:58.471026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509a452-b76c-4d85-b47d-60769a739666 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:58.471399 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509a452-b76c-4d85-b47d-60769a739666 admin admin] PATH_INFO: `/v3/projects/7bda058f10df4cf0855073bf75b4e9d4` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:58.472807 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509a452-b76c-4d85-b47d-60769a739666 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bda058f10df4cf0855073bf75b4e9d4)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:58.474860 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509a452-b76c-4d85-b47d-60769a739666 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:58.526778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d7e56888-6bf6-4551-a8a0-2020529998e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f914ba072a714559b820e9db6c9e9f44', 'request_id': 'req-d7e56888-6bf6-4551-a8a0-2020529998e0'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.535120 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d7e56888-6bf6-4551-a8a0-2020529998e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f914ba072a714559b820e9db6c9e9f44', 'request_id': 'req-d7e56888-6bf6-4551-a8a0-2020529998e0'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.535535 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b509a452-b76c-4d85-b47d-60769a739666 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bda058f10df4cf0855073bf75b4e9d4', 'request_id': 'req-b509a452-b76c-4d85-b47d-60769a739666'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.541199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d7e56888-6bf6-4551-a8a0-2020529998e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f914ba072a714559b820e9db6c9e9f44', 'request_id': 'req-d7e56888-6bf6-4551-a8a0-2020529998e0'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.543842 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b509a452-b76c-4d85-b47d-60769a739666 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bda058f10df4cf0855073bf75b4e9d4', 'request_id': 'req-b509a452-b76c-4d85-b47d-60769a739666'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.556788 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d7e56888-6bf6-4551-a8a0-2020529998e0 admin admin] The token cache is being invalidate because project f914ba072a714559b820e9db6c9e9f44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:58.557205 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d7e56888-6bf6-4551-a8a0-2020529998e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.557399 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b509a452-b76c-4d85-b47d-60769a739666 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bda058f10df4cf0855073bf75b4e9d4', 'request_id': 'req-b509a452-b76c-4d85-b47d-60769a739666'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.558305 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d7e56888-6bf6-4551-a8a0-2020529998e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.560466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 951/1911] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:58 2022] DELETE /identity/v3/projects/f914ba072a714559b820e9db6c9e9f44 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:58.561610 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e56888-6bf6-4551-a8a0-2020529998e0 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:58.563314 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d7e56888-6bf6-4551-a8a0-2020529998e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:58.573746 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b509a452-b76c-4d85-b47d-60769a739666 admin admin] The token cache is being invalidate because project 7bda058f10df4cf0855073bf75b4e9d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:58.574106 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b509a452-b76c-4d85-b47d-60769a739666 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.575399 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b509a452-b76c-4d85-b47d-60769a739666 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.577689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 961/1912] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:58 2022] DELETE /identity/v3/projects/7bda058f10df4cf0855073bf75b4e9d4 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:58.579971 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc1b22-ce28-4d31-9422-de4670e9b069 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:58.580357 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc1b22-ce28-4d31-9422-de4670e9b069 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:58.580731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc1b22-ce28-4d31-9422-de4670e9b069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:58.625553 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f0cc1b22-ce28-4d31-9422-de4670e9b069 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:58.625553 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f0cc1b22-ce28-4d31-9422-de4670e9b069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:58.692500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e56888-6bf6-4551-a8a0-2020529998e0 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:58.698151 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c7c98-3c5f-465d-9711-b155875a0234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:58.699576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c7c98-3c5f-465d-9711-b155875a0234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:58.700041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c7c98-3c5f-465d-9711-b155875a0234 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:58.700145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 962/1913] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:58.700732 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c7c98-3c5f-465d-9711-b155875a0234 admin admin] PATH_INFO: `/v3/projects/266aca812e954458b119d02d32af7b9b` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:58.700932 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc1b22-ce28-4d31-9422-de4670e9b069 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:58.702329 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c7c98-3c5f-465d-9711-b155875a0234 admin admin] RBAC: Authorizing `identity:delete_project(project_id=266aca812e954458b119d02d32af7b9b)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:58.706146 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c7c98-3c5f-465d-9711-b155875a0234 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:58.708990 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f0cc1b22-ce28-4d31-9422-de4670e9b069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:58.725479 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc1b22-ce28-4d31-9422-de4670e9b069 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:58.730520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6ada4-4fd2-4f06-bb43-7d747a530744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:58.731937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6ada4-4fd2-4f06-bb43-7d747a530744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:58.732491 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6ada4-4fd2-4f06-bb43-7d747a530744 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:58.732787 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6ada4-4fd2-4f06-bb43-7d747a530744 admin admin] PATH_INFO: `/v3/users/1c91f1634db84bc38de28e16436d330c` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:58.734033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6ada4-4fd2-4f06-bb43-7d747a530744 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c91f1634db84bc38de28e16436d330c)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:58.736026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6ada4-4fd2-4f06-bb43-7d747a530744 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:58.781478 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-859c7c98-3c5f-465d-9711-b155875a0234 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '266aca812e954458b119d02d32af7b9b', 'request_id': 'req-859c7c98-3c5f-465d-9711-b155875a0234'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.789616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-859c7c98-3c5f-465d-9711-b155875a0234 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '266aca812e954458b119d02d32af7b9b', 'request_id': 'req-859c7c98-3c5f-465d-9711-b155875a0234'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.796816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-859c7c98-3c5f-465d-9711-b155875a0234 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '266aca812e954458b119d02d32af7b9b', 'request_id': 'req-859c7c98-3c5f-465d-9711-b155875a0234'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.798040 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eba6ada4-4fd2-4f06-bb43-7d747a530744 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c91f1634db84bc38de28e16436d330c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.808012 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eba6ada4-4fd2-4f06-bb43-7d747a530744 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c91f1634db84bc38de28e16436d330c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.814564 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-859c7c98-3c5f-465d-9711-b155875a0234 admin admin] The token cache is being invalidate because project 266aca812e954458b119d02d32af7b9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:58.814564 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-859c7c98-3c5f-465d-9711-b155875a0234 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.815416 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-859c7c98-3c5f-465d-9711-b155875a0234 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.817290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 952/1914] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:58 2022] DELETE /identity/v3/projects/266aca812e954458b119d02d32af7b9b => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:58.819136 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aadfb8-0e21-420f-b2e6-050d678c4885 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:58.819362 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aadfb8-0e21-420f-b2e6-050d678c4885 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:58.819645 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aadfb8-0e21-420f-b2e6-050d678c4885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:58.822171 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eba6ada4-4fd2-4f06-bb43-7d747a530744 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c91f1634db84bc38de28e16436d330c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.822794 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eba6ada4-4fd2-4f06-bb43-7d747a530744 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c91f1634db84bc38de28e16436d330c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.836013 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eba6ada4-4fd2-4f06-bb43-7d747a530744 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c91f1634db84bc38de28e16436d330c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.841667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-c7aadfb8-0e21-420f-b2e6-050d678c4885 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:58.845338 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 963/1915] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:02:58 2022] DELETE /identity/v3/users/1c91f1634db84bc38de28e16436d330c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:58.845480 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c7aadfb8-0e21-420f-b2e6-050d678c4885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:58.845560 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6ada4-4fd2-4f06-bb43-7d747a530744 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:58.845560 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-eba6ada4-4fd2-4f06-bb43-7d747a530744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:58.910482 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 953/1916] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:58.912249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed55351-62fc-4081-b5d4-02f50b928ea9 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:58.912626 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed55351-62fc-4081-b5d4-02f50b928ea9 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:58.912971 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed55351-62fc-4081-b5d4-02f50b928ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:58.917276 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6ada4-4fd2-4f06-bb43-7d747a530744 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:58.922338 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc969dc-f1e7-4ab6-9205-ec4918904e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:58.923199 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc969dc-f1e7-4ab6-9205-ec4918904e5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:58.923607 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc969dc-f1e7-4ab6-9205-ec4918904e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:58.923886 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc969dc-f1e7-4ab6-9205-ec4918904e5d admin admin] PATH_INFO: `/v3/projects/b5dfe9f2f52e431fbe01f9d4ffaef46c` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:58.925067 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc969dc-f1e7-4ab6-9205-ec4918904e5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5dfe9f2f52e431fbe01f9d4ffaef46c)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:58.927113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc969dc-f1e7-4ab6-9205-ec4918904e5d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:58.931755 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-bed55351-62fc-4081-b5d4-02f50b928ea9 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:58.932956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-bed55351-62fc-4081-b5d4-02f50b928ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:58.945638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 954/1917] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:58.952717 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed55351-62fc-4081-b5d4-02f50b928ea9 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:58.965880 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed55351-62fc-4081-b5d4-02f50b928ea9 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:58.979497 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0fc969dc-f1e7-4ab6-9205-ec4918904e5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5dfe9f2f52e431fbe01f9d4ffaef46c', 'request_id': 'req-0fc969dc-f1e7-4ab6-9205-ec4918904e5d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.985754 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0fc969dc-f1e7-4ab6-9205-ec4918904e5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5dfe9f2f52e431fbe01f9d4ffaef46c', 'request_id': 'req-0fc969dc-f1e7-4ab6-9205-ec4918904e5d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:58.996164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0fc969dc-f1e7-4ab6-9205-ec4918904e5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5dfe9f2f52e431fbe01f9d4ffaef46c', 'request_id': 'req-0fc969dc-f1e7-4ab6-9205-ec4918904e5d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:59.013059 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0fc969dc-f1e7-4ab6-9205-ec4918904e5d admin admin] The token cache is being invalidate because project b5dfe9f2f52e431fbe01f9d4ffaef46c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:59.013680 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0fc969dc-f1e7-4ab6-9205-ec4918904e5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:59.014514 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0fc969dc-f1e7-4ab6-9205-ec4918904e5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:59.016816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 964/1918] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:58 2022] DELETE /identity/v3/projects/b5dfe9f2f52e431fbe01f9d4ffaef46c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:59.021364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9e90c-2189-4406-b2ba-4c4a8d5e10e9 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.021364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9e90c-2189-4406-b2ba-4c4a8d5e10e9 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.021364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9e90c-2189-4406-b2ba-4c4a8d5e10e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.036909 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-fff9e90c-2189-4406-b2ba-4c4a8d5e10e9 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:59.038144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-fff9e90c-2189-4406-b2ba-4c4a8d5e10e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:59.044807 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2039ce1b-c29b-4803-92f8-4c502a7ae2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:59.046014 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2039ce1b-c29b-4803-92f8-4c502a7ae2dd admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.046014 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2039ce1b-c29b-4803-92f8-4c502a7ae2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.046014 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2039ce1b-c29b-4803-92f8-4c502a7ae2dd admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.046730 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2039ce1b-c29b-4803-92f8-4c502a7ae2dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:02:59.047031 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2039ce1b-c29b-4803-92f8-4c502a7ae2dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:59.048834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2039ce1b-c29b-4803-92f8-4c502a7ae2dd admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:59.065194 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 955/1919] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:02:59.066700 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a386c6b-f545-4b95-bd15-b7c0ad5b2f26 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.067075 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a386c6b-f545-4b95-bd15-b7c0ad5b2f26 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.067420 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a386c6b-f545-4b95-bd15-b7c0ad5b2f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.089487 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-5a386c6b-f545-4b95-bd15-b7c0ad5b2f26 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:59.090832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5a386c6b-f545-4b95-bd15-b7c0ad5b2f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:59.105513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 956/1920] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:59.105513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a386c6b-f545-4b95-bd15-b7c0ad5b2f26 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:59.106850 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5a386c6b-f545-4b95-bd15-b7c0ad5b2f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:59.120756 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 965/1921] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:59.120756 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0167c-a7b2-4461-b98c-3d130b722334 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.120756 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0167c-a7b2-4461-b98c-3d130b722334 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.120756 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0167c-a7b2-4461-b98c-3d130b722334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.121175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a386c6b-f545-4b95-bd15-b7c0ad5b2f26 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:59.125143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca23263-ffe6-4619-9228-b250a5e7edcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:59.126507 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca23263-ffe6-4619-9228-b250a5e7edcf admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.126831 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca23263-ffe6-4619-9228-b250a5e7edcf admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.127178 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca23263-ffe6-4619-9228-b250a5e7edcf admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.127994 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca23263-ffe6-4619-9228-b250a5e7edcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:59.129983 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca23263-ffe6-4619-9228-b250a5e7edcf admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:59.141731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-12a0167c-a7b2-4461-b98c-3d130b722334 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:59.143466 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-12a0167c-a7b2-4461-b98c-3d130b722334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:59.155941 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 966/1922] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:59.158381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0167c-a7b2-4461-b98c-3d130b722334 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:59.166613 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 957/1923] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:59 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:59.167430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca23263-ffe6-4619-9228-b250a5e7edcf admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:59.171329 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7ca23263-ffe6-4619-9228-b250a5e7edcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:59.178737 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0167c-a7b2-4461-b98c-3d130b722334 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:59.247011 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca23263-ffe6-4619-9228-b250a5e7edcf admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:59.248938 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba7151-c82a-4c55-9226-843e90c719cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:59.249692 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba7151-c82a-4c55-9226-843e90c719cd admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.250099 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba7151-c82a-4c55-9226-843e90c719cd admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.251138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba7151-c82a-4c55-9226-843e90c719cd admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.251934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba7151-c82a-4c55-9226-843e90c719cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:02:59.252195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e03b44-51ba-44b8-9382-c17943891d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:59.252301 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba7151-c82a-4c55-9226-843e90c719cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:59.253048 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e03b44-51ba-44b8-9382-c17943891d96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.253322 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e03b44-51ba-44b8-9382-c17943891d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.253586 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e03b44-51ba-44b8-9382-c17943891d96 admin admin] PATH_INFO: `/v3/projects/23c33e50fa7e493197afcde22ca11bd0` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.254225 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba7151-c82a-4c55-9226-843e90c719cd admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:59.254801 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e03b44-51ba-44b8-9382-c17943891d96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23c33e50fa7e493197afcde22ca11bd0)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:59.256699 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e03b44-51ba-44b8-9382-c17943891d96 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:59.266338 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 967/1924] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:02:59.267255 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba7151-c82a-4c55-9226-843e90c719cd admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:59.277531 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba7151-c82a-4c55-9226-843e90c719cd admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:59.283023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497dc33d-0a20-45eb-9580-ffa08dee63cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:59.283711 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dc33d-0a20-45eb-9580-ffa08dee63cc admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.284020 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dc33d-0a20-45eb-9580-ffa08dee63cc admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.284318 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dc33d-0a20-45eb-9580-ffa08dee63cc admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.285261 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497dc33d-0a20-45eb-9580-ffa08dee63cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:02:59.285586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497dc33d-0a20-45eb-9580-ffa08dee63cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:59.287403 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497dc33d-0a20-45eb-9580-ffa08dee63cc admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:59.300958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 968/1925] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:02:59.301775 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497dc33d-0a20-45eb-9580-ffa08dee63cc admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:59.312179 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-87e03b44-51ba-44b8-9382-c17943891d96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23c33e50fa7e493197afcde22ca11bd0', 'request_id': 'req-87e03b44-51ba-44b8-9382-c17943891d96'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:59.312539 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497dc33d-0a20-45eb-9580-ffa08dee63cc admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:59.320892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-87e03b44-51ba-44b8-9382-c17943891d96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23c33e50fa7e493197afcde22ca11bd0', 'request_id': 'req-87e03b44-51ba-44b8-9382-c17943891d96'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:59.326307 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-87e03b44-51ba-44b8-9382-c17943891d96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23c33e50fa7e493197afcde22ca11bd0', 'request_id': 'req-87e03b44-51ba-44b8-9382-c17943891d96'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:59.339093 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-87e03b44-51ba-44b8-9382-c17943891d96 admin admin] The token cache is being invalidate because project 23c33e50fa7e493197afcde22ca11bd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:02:59.339379 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-87e03b44-51ba-44b8-9382-c17943891d96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:59.340016 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-87e03b44-51ba-44b8-9382-c17943891d96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:02:59.341572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 958/1926] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:02:59 2022] DELETE /identity/v3/projects/23c33e50fa7e493197afcde22ca11bd0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:59.342338 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e03b44-51ba-44b8-9382-c17943891d96 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:59.343619 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-87e03b44-51ba-44b8-9382-c17943891d96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:59.380956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fbca2-ccb3-4667-a339-9740bdec93d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:59.381635 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fbca2-ccb3-4667-a339-9740bdec93d4 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.381919 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fbca2-ccb3-4667-a339-9740bdec93d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.382226 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fbca2-ccb3-4667-a339-9740bdec93d4 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.382911 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fbca2-ccb3-4667-a339-9740bdec93d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:59.384821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fbca2-ccb3-4667-a339-9740bdec93d4 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:59.404441 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 969/1927] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:59 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:59.405230 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fbca2-ccb3-4667-a339-9740bdec93d4 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:59.405415 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e03b44-51ba-44b8-9382-c17943891d96 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:59.409363 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-649fbca2-ccb3-4667-a339-9740bdec93d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:59.412195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da64cbe-5807-4759-99e1-f134dbc95ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:59.412875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da64cbe-5807-4759-99e1-f134dbc95ad1 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.413152 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da64cbe-5807-4759-99e1-f134dbc95ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.413424 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da64cbe-5807-4759-99e1-f134dbc95ad1 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.414173 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da64cbe-5807-4759-99e1-f134dbc95ad1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:59.416402 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da64cbe-5807-4759-99e1-f134dbc95ad1 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:59.421096 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fbca2-ccb3-4667-a339-9740bdec93d4 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:59.426199 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e21bf3-cde1-4f2d-a1b7-30d67b31df2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:59.426874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e21bf3-cde1-4f2d-a1b7-30d67b31df2d admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.427153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e21bf3-cde1-4f2d-a1b7-30d67b31df2d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.427426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e21bf3-cde1-4f2d-a1b7-30d67b31df2d admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.428100 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e21bf3-cde1-4f2d-a1b7-30d67b31df2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:59.429987 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e21bf3-cde1-4f2d-a1b7-30d67b31df2d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:59.441287 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 959/1928] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:59 2022] POST /identity/v3/projects => generated 409 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:59.443555 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61715a-388c-4b76-851f-a2f0374d6886 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.443856 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61715a-388c-4b76-851f-a2f0374d6886 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.444426 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61715a-388c-4b76-851f-a2f0374d6886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.459089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 970/1929] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:59 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:59.459089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e21bf3-cde1-4f2d-a1b7-30d67b31df2d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:59.459089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-34e21bf3-cde1-4f2d-a1b7-30d67b31df2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:59.464906 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-1b61715a-388c-4b76-851f-a2f0374d6886 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:59.466051 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1b61715a-388c-4b76-851f-a2f0374d6886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:59.519789 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e21bf3-cde1-4f2d-a1b7-30d67b31df2d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:59.523782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 960/1930] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:59.524368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61715a-388c-4b76-851f-a2f0374d6886 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:59.524605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318355a-a092-42a1-8616-af95c77ab274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:59.525264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318355a-a092-42a1-8616-af95c77ab274 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.525538 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318355a-a092-42a1-8616-af95c77ab274 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.525806 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318355a-a092-42a1-8616-af95c77ab274 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.526556 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318355a-a092-42a1-8616-af95c77ab274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:59.528135 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318355a-a092-42a1-8616-af95c77ab274 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:59.533719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61715a-388c-4b76-851f-a2f0374d6886 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:59.538239 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 971/1931] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:59.538351 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5875b3e-fb84-4d44-82f8-b514261a8b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:59.539013 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318355a-a092-42a1-8616-af95c77ab274 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:59.539140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5875b3e-fb84-4d44-82f8-b514261a8b86 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.539366 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5875b3e-fb84-4d44-82f8-b514261a8b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.539646 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5875b3e-fb84-4d44-82f8-b514261a8b86 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.540324 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5875b3e-fb84-4d44-82f8-b514261a8b86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:59.542299 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5875b3e-fb84-4d44-82f8-b514261a8b86 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:59.548738 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318355a-a092-42a1-8616-af95c77ab274 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:59.553825 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c75d2ae-8f2f-43d8-a6ee-60baf5f65a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:59.554538 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75d2ae-8f2f-43d8-a6ee-60baf5f65a5b admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.554855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75d2ae-8f2f-43d8-a6ee-60baf5f65a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.555136 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75d2ae-8f2f-43d8-a6ee-60baf5f65a5b admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.555819 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c75d2ae-8f2f-43d8-a6ee-60baf5f65a5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:59.557750 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c75d2ae-8f2f-43d8-a6ee-60baf5f65a5b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:59.561479 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 961/1932] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:59 2022] POST /identity/v3/users => generated 305 bytes in 37 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:59.565283 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d82c5-eb8f-406a-92b6-76901225490e None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.565283 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d82c5-eb8f-406a-92b6-76901225490e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.565283 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d82c5-eb8f-406a-92b6-76901225490e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.580239 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 972/1933] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:59 2022] POST /identity/v3/users => generated 292 bytes in 38 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:59.580239 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c75d2ae-8f2f-43d8-a6ee-60baf5f65a5b admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:59.584542 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e20d82c5-eb8f-406a-92b6-76901225490e None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:59.585723 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e20d82c5-eb8f-406a-92b6-76901225490e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:59.587515 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c75d2ae-8f2f-43d8-a6ee-60baf5f65a5b admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:59.593414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b95545-1639-470f-b5ee-fb7ccb318b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:59.594062 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b95545-1639-470f-b5ee-fb7ccb318b28 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.594392 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b95545-1639-470f-b5ee-fb7ccb318b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.594670 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b95545-1639-470f-b5ee-fb7ccb318b28 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.595375 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b95545-1639-470f-b5ee-fb7ccb318b28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:59.596964 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b95545-1639-470f-b5ee-fb7ccb318b28 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:59.598376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 962/1934] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:59.599203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d82c5-eb8f-406a-92b6-76901225490e None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:59.607664 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 973/1935] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:59.608461 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b95545-1639-470f-b5ee-fb7ccb318b28 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:59.608563 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d82c5-eb8f-406a-92b6-76901225490e None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:59.613316 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84cdbe-7fde-4855-aed9-61b24fcd8d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:59.613966 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84cdbe-7fde-4855-aed9-61b24fcd8d72 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.614286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84cdbe-7fde-4855-aed9-61b24fcd8d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.614559 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84cdbe-7fde-4855-aed9-61b24fcd8d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.615276 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c84cdbe-7fde-4855-aed9-61b24fcd8d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:59.616998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c84cdbe-7fde-4855-aed9-61b24fcd8d72 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:59.618537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b95545-1639-470f-b5ee-fb7ccb318b28 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:59.624115 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b5627-8502-4205-9ec5-70e39febfecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:59.624787 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b5627-8502-4205-9ec5-70e39febfecc admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.625076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b5627-8502-4205-9ec5-70e39febfecc admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.625858 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b5627-8502-4205-9ec5-70e39febfecc admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.626697 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b5627-8502-4205-9ec5-70e39febfecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:59.628352 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b5627-8502-4205-9ec5-70e39febfecc admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:59.629203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 963/1936] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:59.630086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84cdbe-7fde-4855-aed9-61b24fcd8d72 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:59.643094 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84cdbe-7fde-4855-aed9-61b24fcd8d72 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:59.644099 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 974/1937] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:59.645170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b5627-8502-4205-9ec5-70e39febfecc admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:59.648870 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522886bb-a2da-4569-b01e-cd6807ba8290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:59.649883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522886bb-a2da-4569-b01e-cd6807ba8290 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.650191 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522886bb-a2da-4569-b01e-cd6807ba8290 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.650464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522886bb-a2da-4569-b01e-cd6807ba8290 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.652786 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522886bb-a2da-4569-b01e-cd6807ba8290 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:02:59.652981 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522886bb-a2da-4569-b01e-cd6807ba8290 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:59.654587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522886bb-a2da-4569-b01e-cd6807ba8290 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:59.655265 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b5627-8502-4205-9ec5-70e39febfecc admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:59.660139 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05541d0-55c2-48b0-ac55-a33fefe1844d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:59.661006 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05541d0-55c2-48b0-ac55-a33fefe1844d admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.661283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05541d0-55c2-48b0-ac55-a33fefe1844d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.661555 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05541d0-55c2-48b0-ac55-a33fefe1844d admin admin] PATH_INFO: `/v3/projects/225886933ebb4238bfa5ed97514ff0f9/users/2f3aab2064584c95949ee6ce258cb16b/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.667702 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 964/1938] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:02:59.669056 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522886bb-a2da-4569-b01e-cd6807ba8290 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:59.677596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05541d0-55c2-48b0-ac55-a33fefe1844d admin admin] RBAC: Authorizing `identity:create_grant(project_id=225886933ebb4238bfa5ed97514ff0f9, user_id=2f3aab2064584c95949ee6ce258cb16b, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:59.680340 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05541d0-55c2-48b0-ac55-a33fefe1844d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:59.680448 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522886bb-a2da-4569-b01e-cd6807ba8290 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:59.686102 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5d24c-9fbf-47e5-b165-0ebc7c257a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:59.686761 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5d24c-9fbf-47e5-b165-0ebc7c257a7d admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.687068 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5d24c-9fbf-47e5-b165-0ebc7c257a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.687338 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5d24c-9fbf-47e5-b165-0ebc7c257a7d admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.688592 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5d24c-9fbf-47e5-b165-0ebc7c257a7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:59.690384 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5d24c-9fbf-47e5-b165-0ebc7c257a7d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:59.695224 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 975/1939] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:59 2022] PUT /identity/v3/projects/225886933ebb4238bfa5ed97514ff0f9/users/2f3aab2064584c95949ee6ce258cb16b/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:59.696146 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05541d0-55c2-48b0-ac55-a33fefe1844d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:59.707891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 965/1940] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:59.709096 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5d24c-9fbf-47e5-b165-0ebc7c257a7d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:59.722676 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5d24c-9fbf-47e5-b165-0ebc7c257a7d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:59.767069 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05541d0-55c2-48b0-ac55-a33fefe1844d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:59.771993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4335e-a9c6-470a-945f-5c55ff1ceb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:59.772667 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4335e-a9c6-470a-945f-5c55ff1ceb9e admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.772946 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4335e-a9c6-470a-945f-5c55ff1ceb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.773340 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4335e-a9c6-470a-945f-5c55ff1ceb9e admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.773960 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd4335e-a9c6-470a-945f-5c55ff1ceb9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:59.775584 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd4335e-a9c6-470a-945f-5c55ff1ceb9e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:59.785642 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 976/1941] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:02:59.787150 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19711ae2-9140-4c82-80ce-ba5c95a46d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:59.787823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19711ae2-9140-4c82-80ce-ba5c95a46d4e admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.788106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19711ae2-9140-4c82-80ce-ba5c95a46d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.788955 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19711ae2-9140-4c82-80ce-ba5c95a46d4e admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.789483 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19711ae2-9140-4c82-80ce-ba5c95a46d4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:59.789999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d8884-a8ce-4313-8d87-eb07c272cc20 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.790314 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d8884-a8ce-4313-8d87-eb07c272cc20 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.790595 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d8884-a8ce-4313-8d87-eb07c272cc20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.791435 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19711ae2-9140-4c82-80ce-ba5c95a46d4e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:59.818982 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-a87d8884-a8ce-4313-8d87-eb07c272cc20 None None] MFA Rules not processed for user `2f3aab2064584c95949ee6ce258cb16b`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:02:59.819367 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 966/1942] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:02:59 2022] POST /identity/v3/projects => generated 391 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:02:59.820170 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19711ae2-9140-4c82-80ce-ba5c95a46d4e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:59.820272 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a87d8884-a8ce-4313-8d87-eb07c272cc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:02:59.886664 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19711ae2-9140-4c82-80ce-ba5c95a46d4e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:59.887785 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 977/1943] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:02:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:02:59.888688 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d8884-a8ce-4313-8d87-eb07c272cc20 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:59.895590 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3567a-dbca-4b75-8bdd-ac5bcef962ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:59.896563 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3567a-dbca-4b75-8bdd-ac5bcef962ef admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.896921 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3567a-dbca-4b75-8bdd-ac5bcef962ef admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.897637 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3567a-dbca-4b75-8bdd-ac5bcef962ef admin admin] PATH_INFO: `/v3/projects/5d40f14dc6f74873ad5f5d10773f5659/users/7aa9b2ebcd574a54b1bf956ed3dc9c23/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.903400 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d8884-a8ce-4313-8d87-eb07c272cc20 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:02:59.908784 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c781fd0-51e3-403d-aa1b-bceeeb9f5828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:02:59.909659 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c781fd0-51e3-403d-aa1b-bceeeb9f5828 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:02:59.909932 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c781fd0-51e3-403d-aa1b-bceeeb9f5828 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:02:59.910227 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c781fd0-51e3-403d-aa1b-bceeeb9f5828 admin admin] PATH_INFO: `/v3/projects/91ec553ce72c4367b27985cfc1918e19/users/ed0a95e385dd4dfb962452465668cc3f/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:02:59.914281 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a3567a-dbca-4b75-8bdd-ac5bcef962ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d40f14dc6f74873ad5f5d10773f5659, user_id=7aa9b2ebcd574a54b1bf956ed3dc9c23, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:59.916928 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a3567a-dbca-4b75-8bdd-ac5bcef962ef admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:59.926377 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c781fd0-51e3-403d-aa1b-bceeeb9f5828 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91ec553ce72c4367b27985cfc1918e19, user_id=ed0a95e385dd4dfb962452465668cc3f, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:02:59.936904 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 967/1944] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:59 2022] PUT /identity/v3/projects/5d40f14dc6f74873ad5f5d10773f5659/users/7aa9b2ebcd574a54b1bf956ed3dc9c23/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:59.936904 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3567a-dbca-4b75-8bdd-ac5bcef962ef admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:59.937123 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c781fd0-51e3-403d-aa1b-bceeeb9f5828 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:02:59.951938 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 978/1945] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:02:59 2022] PUT /identity/v3/projects/91ec553ce72c4367b27985cfc1918e19/users/ed0a95e385dd4dfb962452465668cc3f/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:02:59.952872 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c781fd0-51e3-403d-aa1b-bceeeb9f5828 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:02:59.954622 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5c781fd0-51e3-403d-aa1b-bceeeb9f5828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:00.008482 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3567a-dbca-4b75-8bdd-ac5bcef962ef admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:00.013875 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c781fd0-51e3-403d-aa1b-bceeeb9f5828 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:00.017763 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac162b9-6cc5-4962-952c-82ce033636a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:00.018449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac162b9-6cc5-4962-952c-82ce033636a2 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:00.018732 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac162b9-6cc5-4962-952c-82ce033636a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:00.019009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac162b9-6cc5-4962-952c-82ce033636a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:00.064261 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159bbaa-cc94-4876-ac23-d71207640103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:00.064931 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159bbaa-cc94-4876-ac23-d71207640103 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:00.065211 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159bbaa-cc94-4876-ac23-d71207640103 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:00.065482 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159bbaa-cc94-4876-ac23-d71207640103 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:00.066232 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159bbaa-cc94-4876-ac23-d71207640103 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:00.068198 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159bbaa-cc94-4876-ac23-d71207640103 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:00.075173 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac162b9-6cc5-4962-952c-82ce033636a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:00.076622 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac162b9-6cc5-4962-952c-82ce033636a2 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:00.087367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 979/1946] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:02:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:00.088786 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed547d-7e22-435d-93ec-4285b3e42ed9 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:00.089076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed547d-7e22-435d-93ec-4285b3e42ed9 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:00.089347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed547d-7e22-435d-93ec-4285b3e42ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:00.092246 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 968/1947] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:02:59 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:00.093594 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7f878-5a1f-4220-973d-1f5f8e0ca6ac None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:00.093886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7f878-5a1f-4220-973d-1f5f8e0ca6ac None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:00.094207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7f878-5a1f-4220-973d-1f5f8e0ca6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:00.116812 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-d3ed547d-7e22-435d-93ec-4285b3e42ed9 None None] MFA Rules not processed for user `7aa9b2ebcd574a54b1bf956ed3dc9c23`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:00.118066 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d3ed547d-7e22-435d-93ec-4285b3e42ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:00.123191 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-43a7f878-5a1f-4220-973d-1f5f8e0ca6ac None None] MFA Rules not processed for user `ed0a95e385dd4dfb962452465668cc3f`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:00.124844 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-43a7f878-5a1f-4220-973d-1f5f8e0ca6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:00.191285 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 969/1948] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:00.192135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7f878-5a1f-4220-973d-1f5f8e0ca6ac None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:00.192718 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 980/1949] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:00 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:00.195106 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed547d-7e22-435d-93ec-4285b3e42ed9 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:00.205771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed547d-7e22-435d-93ec-4285b3e42ed9 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:00.207413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7f878-5a1f-4220-973d-1f5f8e0ca6ac None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:00.212992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc07f6-fb68-456b-8c75-21cb2726f7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:00.213649 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc07f6-fb68-456b-8c75-21cb2726f7c3 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:00.213932 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc07f6-fb68-456b-8c75-21cb2726f7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:00.214391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc07f6-fb68-456b-8c75-21cb2726f7c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:00.214680 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35eb09d-0433-453c-b643-5d93deb6e1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:00.215126 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc07f6-fb68-456b-8c75-21cb2726f7c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:00.215340 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35eb09d-0433-453c-b643-5d93deb6e1eb admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:00.215625 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35eb09d-0433-453c-b643-5d93deb6e1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:00.215900 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35eb09d-0433-453c-b643-5d93deb6e1eb admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:00.216597 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35eb09d-0433-453c-b643-5d93deb6e1eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:00.216748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc07f6-fb68-456b-8c75-21cb2726f7c3 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:00.218565 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35eb09d-0433-453c-b643-5d93deb6e1eb admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:00.243630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 970/1950] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:00.243630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc07f6-fb68-456b-8c75-21cb2726f7c3 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:00.256052 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 981/1951] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:00 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:03:00.256220 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc07f6-fb68-456b-8c75-21cb2726f7c3 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:00.256307 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35eb09d-0433-453c-b643-5d93deb6e1eb admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:00.316223 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd5ff0-c114-4ec9-846a-51c824801600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:00.317040 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd5ff0-c114-4ec9-846a-51c824801600 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:00.317339 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd5ff0-c114-4ec9-846a-51c824801600 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:00.317621 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd5ff0-c114-4ec9-846a-51c824801600 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:00.318342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd5ff0-c114-4ec9-846a-51c824801600 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:00.320406 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd5ff0-c114-4ec9-846a-51c824801600 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:00.325705 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35eb09d-0433-453c-b643-5d93deb6e1eb admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:00.331625 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1cde6-3243-4173-9609-96f64f1b0f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:00.333018 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1cde6-3243-4173-9609-96f64f1b0f0b admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:00.333461 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1cde6-3243-4173-9609-96f64f1b0f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:00.333747 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1cde6-3243-4173-9609-96f64f1b0f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:00.334508 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1cde6-3243-4173-9609-96f64f1b0f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:00.336438 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1cde6-3243-4173-9609-96f64f1b0f0b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:00.350654 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 971/1952] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:00 2022] POST /identity/v3/projects => generated 409 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:00.351595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd5ff0-c114-4ec9-846a-51c824801600 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:00.361553 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 982/1953] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:00.365264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1cde6-3243-4173-9609-96f64f1b0f0b admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:00.381601 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1cde6-3243-4173-9609-96f64f1b0f0b admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:00.431203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd5ff0-c114-4ec9-846a-51c824801600 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:00.435868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d59c73-d091-46dd-8031-2475b3e60bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:00.436749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d59c73-d091-46dd-8031-2475b3e60bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:00.437034 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d59c73-d091-46dd-8031-2475b3e60bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:00.437305 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d59c73-d091-46dd-8031-2475b3e60bb4 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:00.438055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d59c73-d091-46dd-8031-2475b3e60bb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:00.439964 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d59c73-d091-46dd-8031-2475b3e60bb4 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:00.448104 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e5a75-5fa8-420d-b064-aa8d4d397f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:00.448782 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e5a75-5fa8-420d-b064-aa8d4d397f02 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:00.449065 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e5a75-5fa8-420d-b064-aa8d4d397f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:00.449344 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e5a75-5fa8-420d-b064-aa8d4d397f02 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:00.450043 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e5a75-5fa8-420d-b064-aa8d4d397f02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:00.452013 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e5a75-5fa8-420d-b064-aa8d4d397f02 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:00.471133 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 972/1954] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:00 2022] POST /identity/v3/users => generated 299 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:00.472256 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d59c73-d091-46dd-8031-2475b3e60bb4 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:00.480153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 983/1955] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:00 2022] POST /identity/v3/users => generated 311 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:00.481882 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e5a75-5fa8-420d-b064-aa8d4d397f02 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:00.490513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d59c73-d091-46dd-8031-2475b3e60bb4 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:00.495057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e5a75-5fa8-420d-b064-aa8d4d397f02 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:00.501232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60aabeb-4cd1-4609-95cd-f3b517b0699c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:00.501562 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60aabeb-4cd1-4609-95cd-f3b517b0699c admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:00.501943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59340d-803e-4527-bd80-c0e3f6ebd6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:00.502352 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60aabeb-4cd1-4609-95cd-f3b517b0699c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:00.502352 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60aabeb-4cd1-4609-95cd-f3b517b0699c admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:00.502932 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59340d-803e-4527-bd80-c0e3f6ebd6df admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:00.503214 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59340d-803e-4527-bd80-c0e3f6ebd6df admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:00.503314 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60aabeb-4cd1-4609-95cd-f3b517b0699c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:00.503961 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59340d-803e-4527-bd80-c0e3f6ebd6df admin admin] PATH_INFO: `/v3/projects/131ff4e1e5664f4b99da227fc07c42d6/users/f1fdc7f2edd643eab90ad78a64f13708/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:00.505546 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60aabeb-4cd1-4609-95cd-f3b517b0699c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:00.517229 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 984/1956] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:00.518151 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60aabeb-4cd1-4609-95cd-f3b517b0699c admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:00.523162 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59340d-803e-4527-bd80-c0e3f6ebd6df admin admin] RBAC: Authorizing `identity:create_grant(project_id=131ff4e1e5664f4b99da227fc07c42d6, user_id=f1fdc7f2edd643eab90ad78a64f13708, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:00.526209 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59340d-803e-4527-bd80-c0e3f6ebd6df admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:00.533201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60aabeb-4cd1-4609-95cd-f3b517b0699c admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:00.540294 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f142e9-ef4d-46d9-af83-b878833c3804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:00.541029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f142e9-ef4d-46d9-af83-b878833c3804 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:00.541441 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f142e9-ef4d-46d9-af83-b878833c3804 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:00.541845 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f142e9-ef4d-46d9-af83-b878833c3804 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:00.542779 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f142e9-ef4d-46d9-af83-b878833c3804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:00.544747 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f142e9-ef4d-46d9-af83-b878833c3804 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:00.550194 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 973/1957] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:00 2022] PUT /identity/v3/projects/131ff4e1e5664f4b99da227fc07c42d6/users/f1fdc7f2edd643eab90ad78a64f13708/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:00.551070 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59340d-803e-4527-bd80-c0e3f6ebd6df admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:00.556205 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 985/1958] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:00.559819 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a92df-408b-4625-a647-1a04746484da None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:00.560179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a92df-408b-4625-a647-1a04746484da None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:00.560532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a92df-408b-4625-a647-1a04746484da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:00.585692 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-575a92df-408b-4625-a647-1a04746484da None None] MFA Rules not processed for user `f1fdc7f2edd643eab90ad78a64f13708`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:00.586960 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-575a92df-408b-4625-a647-1a04746484da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:00.614403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59340d-803e-4527-bd80-c0e3f6ebd6df admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:00.619181 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c912c6-6274-4e17-843c-91566e22b416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:00.620117 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c912c6-6274-4e17-843c-91566e22b416 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:00.620404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c912c6-6274-4e17-843c-91566e22b416 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:00.620679 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c912c6-6274-4e17-843c-91566e22b416 admin admin] PATH_INFO: `/v3/projects/95ac9f70716d49aaaac1fb03684c32ac/users/6052214e89774d66a17e0425bfcb0965/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:00.637039 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c912c6-6274-4e17-843c-91566e22b416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95ac9f70716d49aaaac1fb03684c32ac, user_id=6052214e89774d66a17e0425bfcb0965, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:00.639984 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c912c6-6274-4e17-843c-91566e22b416 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:00.653487 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 986/1959] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:03:00.654332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a92df-408b-4625-a647-1a04746484da None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:00.655099 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 974/1960] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:00 2022] PUT /identity/v3/projects/95ac9f70716d49aaaac1fb03684c32ac/users/6052214e89774d66a17e0425bfcb0965/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:00.658640 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c912c6-6274-4e17-843c-91566e22b416 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:00.668233 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a92df-408b-4625-a647-1a04746484da None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:00.717852 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c912c6-6274-4e17-843c-91566e22b416 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:00.723929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353c3a8-1b9d-43b2-92ce-fa19207bbb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:00.724324 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d707600-5308-4040-b284-ab7793664775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:00.724839 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353c3a8-1b9d-43b2-92ce-fa19207bbb67 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:00.724952 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d707600-5308-4040-b284-ab7793664775 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:00.725114 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353c3a8-1b9d-43b2-92ce-fa19207bbb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:00.725231 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d707600-5308-4040-b284-ab7793664775 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:00.725381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353c3a8-1b9d-43b2-92ce-fa19207bbb67 admin admin] PATH_INFO: `/v3/projects/8dc026a98fd841718f4be06f6e69d016/users/d36350dfa14544e2ae1225c8adb90b17/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:00.725505 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d707600-5308-4040-b284-ab7793664775 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:00.726224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d707600-5308-4040-b284-ab7793664775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:00.728181 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d707600-5308-4040-b284-ab7793664775 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:00.740868 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c353c3a8-1b9d-43b2-92ce-fa19207bbb67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dc026a98fd841718f4be06f6e69d016, user_id=d36350dfa14544e2ae1225c8adb90b17, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:00.743550 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c353c3a8-1b9d-43b2-92ce-fa19207bbb67 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:00.760253 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 975/1961] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:00 2022] POST /identity/v3/projects => generated 393 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:03:00.760438 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 987/1962] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:00 2022] PUT /identity/v3/projects/8dc026a98fd841718f4be06f6e69d016/users/d36350dfa14544e2ae1225c8adb90b17/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:00.760559 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d707600-5308-4040-b284-ab7793664775 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:00.760655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353c3a8-1b9d-43b2-92ce-fa19207bbb67 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:00.825074 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353c3a8-1b9d-43b2-92ce-fa19207bbb67 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:00.828357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d707600-5308-4040-b284-ab7793664775 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:00.831053 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310f05c-f41f-4046-abef-730b0dd2c278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:00.831923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310f05c-f41f-4046-abef-730b0dd2c278 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:00.832210 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310f05c-f41f-4046-abef-730b0dd2c278 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:00.832610 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310f05c-f41f-4046-abef-730b0dd2c278 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:00.833335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310f05c-f41f-4046-abef-730b0dd2c278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:00.833468 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d28ff2-6808-4999-a8aa-0e0c1430405f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:00.834029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d28ff2-6808-4999-a8aa-0e0c1430405f admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:00.834342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d28ff2-6808-4999-a8aa-0e0c1430405f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:00.834614 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d28ff2-6808-4999-a8aa-0e0c1430405f admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:00.835011 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310f05c-f41f-4046-abef-730b0dd2c278 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:00.835332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d28ff2-6808-4999-a8aa-0e0c1430405f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:00.836910 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d28ff2-6808-4999-a8aa-0e0c1430405f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:00.846563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 988/1963] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:00.847547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310f05c-f41f-4046-abef-730b0dd2c278 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:00.850285 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 976/1964] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:00.852514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d28ff2-6808-4999-a8aa-0e0c1430405f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:00.859094 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310f05c-f41f-4046-abef-730b0dd2c278 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:00.863677 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d28ff2-6808-4999-a8aa-0e0c1430405f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:00.864771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3710b-08be-4fc9-ba09-f4e8622f9b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:00.865445 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3710b-08be-4fc9-ba09-f4e8622f9b50 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:00.865724 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3710b-08be-4fc9-ba09-f4e8622f9b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:00.866000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3710b-08be-4fc9-ba09-f4e8622f9b50 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:00.866719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3710b-08be-4fc9-ba09-f4e8622f9b50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:00.868727 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3710b-08be-4fc9-ba09-f4e8622f9b50 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:00.868928 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50019360-3e2e-4dfa-8214-5c350d619bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:00.869835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50019360-3e2e-4dfa-8214-5c350d619bd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:00.870075 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50019360-3e2e-4dfa-8214-5c350d619bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:00.870377 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50019360-3e2e-4dfa-8214-5c350d619bd6 admin admin] PATH_INFO: `/v3/projects/8dc026a98fd841718f4be06f6e69d016/users/d36350dfa14544e2ae1225c8adb90b17/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:00.872320 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50019360-3e2e-4dfa-8214-5c350d619bd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dc026a98fd841718f4be06f6e69d016, user_id=d36350dfa14544e2ae1225c8adb90b17, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:00.875015 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50019360-3e2e-4dfa-8214-5c350d619bd6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:00.891396 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 977/1965] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:00 2022] PUT /identity/v3/projects/8dc026a98fd841718f4be06f6e69d016/users/d36350dfa14544e2ae1225c8adb90b17/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:00.892164 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50019360-3e2e-4dfa-8214-5c350d619bd6 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:00.898786 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 989/1966] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:00 2022] POST /identity/v3/users => generated 303 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:00.902473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36475ce9-a6fe-473b-b00d-e13346246a36 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:00.902875 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36475ce9-a6fe-473b-b00d-e13346246a36 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:00.903552 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36475ce9-a6fe-473b-b00d-e13346246a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:00.932117 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-36475ce9-a6fe-473b-b00d-e13346246a36 None None] MFA Rules not processed for user `d36350dfa14544e2ae1225c8adb90b17`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:00.933338 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-36475ce9-a6fe-473b-b00d-e13346246a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:00.954204 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50019360-3e2e-4dfa-8214-5c350d619bd6 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:00.960119 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e8c24-bdd7-40b1-86d3-68dc9c47ce03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:00.961016 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e8c24-bdd7-40b1-86d3-68dc9c47ce03 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:00.961294 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e8c24-bdd7-40b1-86d3-68dc9c47ce03 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:00.961562 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e8c24-bdd7-40b1-86d3-68dc9c47ce03 admin admin] PATH_INFO: `/v3/projects/95ac9f70716d49aaaac1fb03684c32ac/users/6052214e89774d66a17e0425bfcb0965/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:00.963599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e8c24-bdd7-40b1-86d3-68dc9c47ce03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95ac9f70716d49aaaac1fb03684c32ac, user_id=6052214e89774d66a17e0425bfcb0965, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:00.967000 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e8c24-bdd7-40b1-86d3-68dc9c47ce03 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:00.977940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 978/1967] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:00 2022] PUT /identity/v3/projects/95ac9f70716d49aaaac1fb03684c32ac/users/6052214e89774d66a17e0425bfcb0965/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:00.978774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e8c24-bdd7-40b1-86d3-68dc9c47ce03 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:01.040752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 990/1968] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:00 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:01.042159 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e74066-e1bb-45fc-9dd9-4b54ec9e348b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:01.042463 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e74066-e1bb-45fc-9dd9-4b54ec9e348b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:01.042745 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e74066-e1bb-45fc-9dd9-4b54ec9e348b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:01.062107 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e8c24-bdd7-40b1-86d3-68dc9c47ce03 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:01.067500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c50e2-44e5-41a3-9b19-1a3c81c60203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:01.068322 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c50e2-44e5-41a3-9b19-1a3c81c60203 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:01.068901 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c50e2-44e5-41a3-9b19-1a3c81c60203 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:01.069178 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c50e2-44e5-41a3-9b19-1a3c81c60203 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:01.069901 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c50e2-44e5-41a3-9b19-1a3c81c60203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:01.071528 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c50e2-44e5-41a3-9b19-1a3c81c60203 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:01.072692 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e7e74066-e1bb-45fc-9dd9-4b54ec9e348b None None] MFA Rules not processed for user `6052214e89774d66a17e0425bfcb0965`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:01.074683 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e7e74066-e1bb-45fc-9dd9-4b54ec9e348b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:01.084998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 979/1969] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:01.084998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c50e2-44e5-41a3-9b19-1a3c81c60203 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:01.152902 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c50e2-44e5-41a3-9b19-1a3c81c60203 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:01.158217 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b57a22-ddcb-419e-97ad-add77a01af29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:01.158868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b57a22-ddcb-419e-97ad-add77a01af29 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:01.159241 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b57a22-ddcb-419e-97ad-add77a01af29 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:01.159520 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b57a22-ddcb-419e-97ad-add77a01af29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:01.176473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 991/1970] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:01.182853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e74066-e1bb-45fc-9dd9-4b54ec9e348b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:01.201372 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e74066-e1bb-45fc-9dd9-4b54ec9e348b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:01.207834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f9a67-e192-4680-8f97-85f07e8ed63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:01.208801 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f9a67-e192-4680-8f97-85f07e8ed63d admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:01.209083 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f9a67-e192-4680-8f97-85f07e8ed63d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:01.209415 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f9a67-e192-4680-8f97-85f07e8ed63d admin admin] PATH_INFO: `/v3/projects/9ec4003f00054f7cb318fe82fb1d6510/users/c9635db01def48df8adb5d4cd6b1b25a/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:01.228090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f9a67-e192-4680-8f97-85f07e8ed63d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ec4003f00054f7cb318fe82fb1d6510, user_id=c9635db01def48df8adb5d4cd6b1b25a, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:01.230767 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f9a67-e192-4680-8f97-85f07e8ed63d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:01.243878 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b57a22-ddcb-419e-97ad-add77a01af29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:01.245426 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b57a22-ddcb-419e-97ad-add77a01af29 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:01.255634 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 992/1971] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:01 2022] PUT /identity/v3/projects/9ec4003f00054f7cb318fe82fb1d6510/users/c9635db01def48df8adb5d4cd6b1b25a/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:01.255810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 980/1972] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:01 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:01.256492 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f9a67-e192-4680-8f97-85f07e8ed63d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:01.260034 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b57a22-ddcb-419e-97ad-add77a01af29 service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:01.261003 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-562f9a67-e192-4680-8f97-85f07e8ed63d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:01.276472 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b57a22-ddcb-419e-97ad-add77a01af29 service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:01.335269 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f9a67-e192-4680-8f97-85f07e8ed63d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:01.343060 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f20cb5-0a8d-4700-b268-358439e92926 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:01.343721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f20cb5-0a8d-4700-b268-358439e92926 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:01.344601 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f20cb5-0a8d-4700-b268-358439e92926 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:01.344880 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f20cb5-0a8d-4700-b268-358439e92926 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:01.365952 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258005e5-89a2-4921-bf7d-bb8d30a5f3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:01.366669 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258005e5-89a2-4921-bf7d-bb8d30a5f3c2 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:01.367151 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258005e5-89a2-4921-bf7d-bb8d30a5f3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:01.367338 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258005e5-89a2-4921-bf7d-bb8d30a5f3c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:01.368042 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258005e5-89a2-4921-bf7d-bb8d30a5f3c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:01.370079 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258005e5-89a2-4921-bf7d-bb8d30a5f3c2 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:01.383265 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 981/1973] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:01.387230 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258005e5-89a2-4921-bf7d-bb8d30a5f3c2 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:01.397842 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258005e5-89a2-4921-bf7d-bb8d30a5f3c2 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:01.407378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffdc61-969d-4dc6-8203-9ea43f638324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:01.408407 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffdc61-969d-4dc6-8203-9ea43f638324 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:01.408596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f20cb5-0a8d-4700-b268-358439e92926 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:01.408747 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffdc61-969d-4dc6-8203-9ea43f638324 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:01.409139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffdc61-969d-4dc6-8203-9ea43f638324 admin admin] PATH_INFO: `/v3/projects/9ec4003f00054f7cb318fe82fb1d6510/users/c9635db01def48df8adb5d4cd6b1b25a/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:01.410304 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f20cb5-0a8d-4700-b268-358439e92926 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:01.412958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecffdc61-969d-4dc6-8203-9ea43f638324 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ec4003f00054f7cb318fe82fb1d6510, user_id=c9635db01def48df8adb5d4cd6b1b25a, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:01.418281 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecffdc61-969d-4dc6-8203-9ea43f638324 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:01.421532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 993/1974] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:01.443128 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 982/1975] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:01 2022] PUT /identity/v3/projects/9ec4003f00054f7cb318fe82fb1d6510/users/c9635db01def48df8adb5d4cd6b1b25a/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:03:01.453316 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997403e-3169-4a81-842d-316eeafe9b3b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:01.454142 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997403e-3169-4a81-842d-316eeafe9b3b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:01.458492 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997403e-3169-4a81-842d-316eeafe9b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:01.515016 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-b997403e-3169-4a81-842d-316eeafe9b3b None None] MFA Rules not processed for user `c9635db01def48df8adb5d4cd6b1b25a`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:01.517525 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b997403e-3169-4a81-842d-316eeafe9b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:01.550765 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffdc61-969d-4dc6-8203-9ea43f638324 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:01.648240 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 994/1976] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:01 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:01.672293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffdc61-969d-4dc6-8203-9ea43f638324 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:01.677042 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f75d15-c240-4550-a4f3-d2fd08f80a1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:01.677811 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f75d15-c240-4550-a4f3-d2fd08f80a1b service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:01.678271 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f75d15-c240-4550-a4f3-d2fd08f80a1b service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:01.678617 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f75d15-c240-4550-a4f3-d2fd08f80a1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:01.740949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f75d15-c240-4550-a4f3-d2fd08f80a1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:01.742778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f75d15-c240-4550-a4f3-d2fd08f80a1b service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:01.753890 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 983/1977] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:01 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:02.046096 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997403e-3169-4a81-842d-316eeafe9b3b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:02.060609 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997403e-3169-4a81-842d-316eeafe9b3b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:02.077391 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a120d-8ed7-4c8b-ad0e-83354a42aeaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:02.078060 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a120d-8ed7-4c8b-ad0e-83354a42aeaa service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:02.078417 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a120d-8ed7-4c8b-ad0e-83354a42aeaa service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:02.079208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a120d-8ed7-4c8b-ad0e-83354a42aeaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:02.094906 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a120d-8ed7-4c8b-ad0e-83354a42aeaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:02.096362 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a120d-8ed7-4c8b-ad0e-83354a42aeaa service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:02.109323 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 995/1978] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:02 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:02.199044 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f75d15-c240-4550-a4f3-d2fd08f80a1b service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:02.214298 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f75d15-c240-4550-a4f3-d2fd08f80a1b service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:02.226844 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9e655-8579-4848-a523-bc68587392f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:02.227664 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9e655-8579-4848-a523-bc68587392f2 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:02.227980 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9e655-8579-4848-a523-bc68587392f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:02.228282 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9e655-8579-4848-a523-bc68587392f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:02.287920 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e9e655-8579-4848-a523-bc68587392f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:02.289375 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e9e655-8579-4848-a523-bc68587392f2 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:02.298839 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 984/1979] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:19.399182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a120d-8ed7-4c8b-ad0e-83354a42aeaa service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:19.496746 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a120d-8ed7-4c8b-ad0e-83354a42aeaa service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:19.504769 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea7835-82b4-4270-9cb9-81105ec8ec22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:19.505921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea7835-82b4-4270-9cb9-81105ec8ec22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:19.506318 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea7835-82b4-4270-9cb9-81105ec8ec22 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:19.506695 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea7835-82b4-4270-9cb9-81105ec8ec22 admin admin] PATH_INFO: `/v3/users/f1fdc7f2edd643eab90ad78a64f13708` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:19.508349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea7835-82b4-4270-9cb9-81105ec8ec22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1fdc7f2edd643eab90ad78a64f13708)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:19.510426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea7835-82b4-4270-9cb9-81105ec8ec22 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:19.576451 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-aaea7835-82b4-4270-9cb9-81105ec8ec22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1fdc7f2edd643eab90ad78a64f13708'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:19.588279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-aaea7835-82b4-4270-9cb9-81105ec8ec22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1fdc7f2edd643eab90ad78a64f13708'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:19.602553 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-aaea7835-82b4-4270-9cb9-81105ec8ec22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1fdc7f2edd643eab90ad78a64f13708'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:19.603775 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-aaea7835-82b4-4270-9cb9-81105ec8ec22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1fdc7f2edd643eab90ad78a64f13708'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:19.640704 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-aaea7835-82b4-4270-9cb9-81105ec8ec22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1fdc7f2edd643eab90ad78a64f13708'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:19.652585 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 996/1980] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:19 2022] DELETE /identity/v3/users/f1fdc7f2edd643eab90ad78a64f13708 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:19.657872 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef208a-a6ce-43af-81bf-c7569f92652d None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:19.658429 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef208a-a6ce-43af-81bf-c7569f92652d None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:19.658607 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef208a-a6ce-43af-81bf-c7569f92652d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:19.686131 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a3ef208a-a6ce-43af-81bf-c7569f92652d None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:19.688210 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a3ef208a-a6ce-43af-81bf-c7569f92652d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:19.845319 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 985/1981] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:19.853043 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea7835-82b4-4270-9cb9-81105ec8ec22 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:19.857635 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-aaea7835-82b4-4270-9cb9-81105ec8ec22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:19.952243 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea7835-82b4-4270-9cb9-81105ec8ec22 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:19.954956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef208a-a6ce-43af-81bf-c7569f92652d None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:19.956831 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a3ef208a-a6ce-43af-81bf-c7569f92652d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:19.957808 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8bff83-49e5-4455-8d5e-b1ec88184089 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:19.958758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8bff83-49e5-4455-8d5e-b1ec88184089 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:19.959111 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8bff83-49e5-4455-8d5e-b1ec88184089 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:19.959455 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8bff83-49e5-4455-8d5e-b1ec88184089 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:19.972243 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8bff83-49e5-4455-8d5e-b1ec88184089 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:19.973818 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8bff83-49e5-4455-8d5e-b1ec88184089 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:19.987501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 997/1982] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:20.028319 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef208a-a6ce-43af-81bf-c7569f92652d None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:20.033349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f3a18-c601-4ae6-b8da-0469c9ae97e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:20.034409 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f3a18-c601-4ae6-b8da-0469c9ae97e0 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:20.034808 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f3a18-c601-4ae6-b8da-0469c9ae97e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:20.035090 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f3a18-c601-4ae6-b8da-0469c9ae97e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:20.037728 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-359f3a18-c601-4ae6-b8da-0469c9ae97e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:20.120440 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f3a18-c601-4ae6-b8da-0469c9ae97e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:20.124203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f3a18-c601-4ae6-b8da-0469c9ae97e0 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:20.141562 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 986/1983] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:19 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 23:03:20.482461 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8bff83-49e5-4455-8d5e-b1ec88184089 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:20.487043 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6d8bff83-49e5-4455-8d5e-b1ec88184089 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:20.507529 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8bff83-49e5-4455-8d5e-b1ec88184089 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:20.512546 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155a8b7-b5cc-487c-be79-f4bc1d4fe41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:20.513479 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155a8b7-b5cc-487c-be79-f4bc1d4fe41c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:20.513975 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155a8b7-b5cc-487c-be79-f4bc1d4fe41c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:20.514550 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155a8b7-b5cc-487c-be79-f4bc1d4fe41c admin admin] PATH_INFO: `/v3/projects/131ff4e1e5664f4b99da227fc07c42d6` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:20.516533 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9155a8b7-b5cc-487c-be79-f4bc1d4fe41c admin admin] RBAC: Authorizing `identity:delete_project(project_id=131ff4e1e5664f4b99da227fc07c42d6)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:20.518820 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9155a8b7-b5cc-487c-be79-f4bc1d4fe41c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:20.585043 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9155a8b7-b5cc-487c-be79-f4bc1d4fe41c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '131ff4e1e5664f4b99da227fc07c42d6', 'request_id': 'req-9155a8b7-b5cc-487c-be79-f4bc1d4fe41c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:20.590772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9155a8b7-b5cc-487c-be79-f4bc1d4fe41c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '131ff4e1e5664f4b99da227fc07c42d6', 'request_id': 'req-9155a8b7-b5cc-487c-be79-f4bc1d4fe41c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:20.597502 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9155a8b7-b5cc-487c-be79-f4bc1d4fe41c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '131ff4e1e5664f4b99da227fc07c42d6', 'request_id': 'req-9155a8b7-b5cc-487c-be79-f4bc1d4fe41c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:20.609869 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9155a8b7-b5cc-487c-be79-f4bc1d4fe41c admin admin] The token cache is being invalidate because project 131ff4e1e5664f4b99da227fc07c42d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:20.610256 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9155a8b7-b5cc-487c-be79-f4bc1d4fe41c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:20.610882 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9155a8b7-b5cc-487c-be79-f4bc1d4fe41c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:20.612512 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 998/1984] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:20 2022] DELETE /identity/v3/projects/131ff4e1e5664f4b99da227fc07c42d6 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:20.616647 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f3a18-c601-4ae6-b8da-0469c9ae97e0 service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:20.630815 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f3a18-c601-4ae6-b8da-0469c9ae97e0 service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:20.685566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc2155-00cf-4008-9739-d645a541818b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:20.686371 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc2155-00cf-4008-9739-d645a541818b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:20.686645 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc2155-00cf-4008-9739-d645a541818b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:20.686956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc2155-00cf-4008-9739-d645a541818b admin admin] PATH_INFO: `/v3/users/c9635db01def48df8adb5d4cd6b1b25a` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:20.688840 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc2155-00cf-4008-9739-d645a541818b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9635db01def48df8adb5d4cd6b1b25a)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:20.690942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc2155-00cf-4008-9739-d645a541818b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:20.780404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-32dc2155-00cf-4008-9739-d645a541818b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9635db01def48df8adb5d4cd6b1b25a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:20.792989 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-32dc2155-00cf-4008-9739-d645a541818b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9635db01def48df8adb5d4cd6b1b25a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:20.793888 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-32dc2155-00cf-4008-9739-d645a541818b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9635db01def48df8adb5d4cd6b1b25a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:20.803725 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-32dc2155-00cf-4008-9739-d645a541818b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9635db01def48df8adb5d4cd6b1b25a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:20.804755 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-32dc2155-00cf-4008-9739-d645a541818b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9635db01def48df8adb5d4cd6b1b25a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:20.830172 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 987/1985] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:20 2022] DELETE /identity/v3/users/c9635db01def48df8adb5d4cd6b1b25a => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:21.343348 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155a8b7-b5cc-487c-be79-f4bc1d4fe41c admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:21.345056 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9155a8b7-b5cc-487c-be79-f4bc1d4fe41c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:21.379337 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155a8b7-b5cc-487c-be79-f4bc1d4fe41c admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:21.448764 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc2155-00cf-4008-9739-d645a541818b admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:21.452036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-32dc2155-00cf-4008-9739-d645a541818b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:21.491777 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21b499-0bab-484a-aa51-6973fcca8edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:21.494069 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21b499-0bab-484a-aa51-6973fcca8edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:21.494373 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21b499-0bab-484a-aa51-6973fcca8edc admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:21.494659 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21b499-0bab-484a-aa51-6973fcca8edc admin admin] PATH_INFO: `/v3/projects/9ec4003f00054f7cb318fe82fb1d6510` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:21.516735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21b499-0bab-484a-aa51-6973fcca8edc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ec4003f00054f7cb318fe82fb1d6510)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:21.519848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21b499-0bab-484a-aa51-6973fcca8edc admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:21.598165 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8e21b499-0bab-484a-aa51-6973fcca8edc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ec4003f00054f7cb318fe82fb1d6510', 'request_id': 'req-8e21b499-0bab-484a-aa51-6973fcca8edc'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:21.605998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8e21b499-0bab-484a-aa51-6973fcca8edc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ec4003f00054f7cb318fe82fb1d6510', 'request_id': 'req-8e21b499-0bab-484a-aa51-6973fcca8edc'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:21.614934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8e21b499-0bab-484a-aa51-6973fcca8edc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ec4003f00054f7cb318fe82fb1d6510', 'request_id': 'req-8e21b499-0bab-484a-aa51-6973fcca8edc'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:21.639001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc2155-00cf-4008-9739-d645a541818b admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:21.653106 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8e21b499-0bab-484a-aa51-6973fcca8edc admin admin] The token cache is being invalidate because project 9ec4003f00054f7cb318fe82fb1d6510 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:21.653358 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8e21b499-0bab-484a-aa51-6973fcca8edc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:21.654903 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8e21b499-0bab-484a-aa51-6973fcca8edc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:21.665470 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 999/1986] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:21 2022] DELETE /identity/v3/projects/9ec4003f00054f7cb318fe82fb1d6510 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:21.673116 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff83fa7-bbf2-4310-901f-84a73b7302e4 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:21.673402 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff83fa7-bbf2-4310-901f-84a73b7302e4 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:21.673865 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff83fa7-bbf2-4310-901f-84a73b7302e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:21.716636 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-9ff83fa7-bbf2-4310-901f-84a73b7302e4 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:21.722652 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9ff83fa7-bbf2-4310-901f-84a73b7302e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:21.751732 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f83e598-3019-4c3c-bf4d-b6ecc4562134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:21.753223 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83e598-3019-4c3c-bf4d-b6ecc4562134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:21.754224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83e598-3019-4c3c-bf4d-b6ecc4562134 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:21.755243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83e598-3019-4c3c-bf4d-b6ecc4562134 admin admin] PATH_INFO: `/v3/users/7aa9b2ebcd574a54b1bf956ed3dc9c23` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:21.756775 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f83e598-3019-4c3c-bf4d-b6ecc4562134 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7aa9b2ebcd574a54b1bf956ed3dc9c23)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:21.759082 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f83e598-3019-4c3c-bf4d-b6ecc4562134 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:21.846390 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4f83e598-3019-4c3c-bf4d-b6ecc4562134 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7aa9b2ebcd574a54b1bf956ed3dc9c23'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:21.848028 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1000/1987] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:21.857999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e1c84-05cf-4c30-bd91-2f64a07f4d6b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:21.858363 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e1c84-05cf-4c30-bd91-2f64a07f4d6b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:21.858980 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e1c84-05cf-4c30-bd91-2f64a07f4d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:21.860160 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4f83e598-3019-4c3c-bf4d-b6ecc4562134 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7aa9b2ebcd574a54b1bf956ed3dc9c23'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:21.860998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4f83e598-3019-4c3c-bf4d-b6ecc4562134 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7aa9b2ebcd574a54b1bf956ed3dc9c23'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:21.876657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4f83e598-3019-4c3c-bf4d-b6ecc4562134 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7aa9b2ebcd574a54b1bf956ed3dc9c23'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:21.877435 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4f83e598-3019-4c3c-bf4d-b6ecc4562134 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7aa9b2ebcd574a54b1bf956ed3dc9c23'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:21.903943 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-900e1c84-05cf-4c30-bd91-2f64a07f4d6b None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:21.915814 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-900e1c84-05cf-4c30-bd91-2f64a07f4d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:21.930554 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 988/1988] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:21 2022] DELETE /identity/v3/users/7aa9b2ebcd574a54b1bf956ed3dc9c23 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:21.930554 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865e8fe-ed1e-4fe1-ac90-4fea1d1eb44f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:21.930554 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865e8fe-ed1e-4fe1-ac90-4fea1d1eb44f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:21.930554 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865e8fe-ed1e-4fe1-ac90-4fea1d1eb44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:21.947552 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-9865e8fe-ed1e-4fe1-ac90-4fea1d1eb44f None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:21.948779 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9865e8fe-ed1e-4fe1-ac90-4fea1d1eb44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:22.028590 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 989/1989] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:22.041408 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865e8fe-ed1e-4fe1-ac90-4fea1d1eb44f None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:22.044463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9865e8fe-ed1e-4fe1-ac90-4fea1d1eb44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:22.051993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1001/1990] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:22.056170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e1c84-05cf-4c30-bd91-2f64a07f4d6b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:22.067104 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e1c84-05cf-4c30-bd91-2f64a07f4d6b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:22.072526 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1073380-5881-4bb3-84ab-6f8b455e8711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:22.073248 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1073380-5881-4bb3-84ab-6f8b455e8711 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:22.073568 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1073380-5881-4bb3-84ab-6f8b455e8711 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:22.073881 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1073380-5881-4bb3-84ab-6f8b455e8711 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:22.074674 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1073380-5881-4bb3-84ab-6f8b455e8711 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:03:22.074982 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1073380-5881-4bb3-84ab-6f8b455e8711 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:22.076888 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1073380-5881-4bb3-84ab-6f8b455e8711 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:22.102901 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1002/1991] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 23:03:22.104041 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1073380-5881-4bb3-84ab-6f8b455e8711 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:22.123310 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1073380-5881-4bb3-84ab-6f8b455e8711 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:22.134797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87e48c-fe05-4545-ac46-126ead0513aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:22.135714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87e48c-fe05-4545-ac46-126ead0513aa admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:22.136019 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87e48c-fe05-4545-ac46-126ead0513aa admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:22.136354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87e48c-fe05-4545-ac46-126ead0513aa admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:22.137639 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87e48c-fe05-4545-ac46-126ead0513aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:22.139955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87e48c-fe05-4545-ac46-126ead0513aa admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:22.147820 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865e8fe-ed1e-4fe1-ac90-4fea1d1eb44f None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:22.153138 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab885b8-5f47-4a05-b3dc-5301c5de9ce7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:22.153138 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab885b8-5f47-4a05-b3dc-5301c5de9ce7 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:22.153138 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab885b8-5f47-4a05-b3dc-5301c5de9ce7 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:22.153489 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab885b8-5f47-4a05-b3dc-5301c5de9ce7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:22.168575 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab885b8-5f47-4a05-b3dc-5301c5de9ce7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:22.170270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab885b8-5f47-4a05-b3dc-5301c5de9ce7 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:22.181373 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 990/1992] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:22.184639 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1003/1993] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:22 2022] POST /identity/v3/projects => generated 389 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:22.195193 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab885b8-5f47-4a05-b3dc-5301c5de9ce7 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:22.210583 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab885b8-5f47-4a05-b3dc-5301c5de9ce7 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:22.279721 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d84dc5-321e-461f-8b2e-91ba149e6519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:22.280393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d84dc5-321e-461f-8b2e-91ba149e6519 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:22.280668 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d84dc5-321e-461f-8b2e-91ba149e6519 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:22.281016 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d84dc5-321e-461f-8b2e-91ba149e6519 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:22.281939 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d84dc5-321e-461f-8b2e-91ba149e6519 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:22.284306 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d84dc5-321e-461f-8b2e-91ba149e6519 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:22.309818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 991/1994] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:22 2022] POST /identity/v3/users => generated 295 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:22.313022 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87e48c-fe05-4545-ac46-126ead0513aa admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:22.324666 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87e48c-fe05-4545-ac46-126ead0513aa admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:22.330503 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a873410-61bd-4c5d-90e9-d78ba969d5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:22.331487 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a873410-61bd-4c5d-90e9-d78ba969d5b8 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:22.331961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a873410-61bd-4c5d-90e9-d78ba969d5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:22.332413 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a873410-61bd-4c5d-90e9-d78ba969d5b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:22.333361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a873410-61bd-4c5d-90e9-d78ba969d5b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:22.335775 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a873410-61bd-4c5d-90e9-d78ba969d5b8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:22.351723 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1004/1995] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:22.353446 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d84dc5-321e-461f-8b2e-91ba149e6519 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:22.365481 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d84dc5-321e-461f-8b2e-91ba149e6519 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:22.370421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70082b8-7afe-4ed4-bb37-34433d4ab93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:22.371310 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70082b8-7afe-4ed4-bb37-34433d4ab93d admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:22.371687 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70082b8-7afe-4ed4-bb37-34433d4ab93d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:22.372034 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70082b8-7afe-4ed4-bb37-34433d4ab93d admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:22.372839 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70082b8-7afe-4ed4-bb37-34433d4ab93d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:22.374563 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70082b8-7afe-4ed4-bb37-34433d4ab93d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:22.385644 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 992/1996] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:22.390060 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a873410-61bd-4c5d-90e9-d78ba969d5b8 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:22.400384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a873410-61bd-4c5d-90e9-d78ba969d5b8 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:22.406749 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29039edc-7d20-4d2c-97ca-5a565b96a80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:22.407642 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29039edc-7d20-4d2c-97ca-5a565b96a80f admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:22.407919 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29039edc-7d20-4d2c-97ca-5a565b96a80f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:22.408188 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29039edc-7d20-4d2c-97ca-5a565b96a80f admin admin] PATH_INFO: `/v3/projects/2f993851a0b246e5b33460244c3bd355/users/4269fd18e2484e29984279dc9873ebb7/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:22.428353 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29039edc-7d20-4d2c-97ca-5a565b96a80f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f993851a0b246e5b33460244c3bd355, user_id=4269fd18e2484e29984279dc9873ebb7, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:22.431635 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29039edc-7d20-4d2c-97ca-5a565b96a80f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:22.446921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1005/1997] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:22 2022] PUT /identity/v3/projects/2f993851a0b246e5b33460244c3bd355/users/4269fd18e2484e29984279dc9873ebb7/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:22.458947 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56c666-c8f3-459b-9dae-17fdcd9ea57a None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:22.458947 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56c666-c8f3-459b-9dae-17fdcd9ea57a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:22.458947 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56c666-c8f3-459b-9dae-17fdcd9ea57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:22.483837 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29039edc-7d20-4d2c-97ca-5a565b96a80f admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:22.488510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-29039edc-7d20-4d2c-97ca-5a565b96a80f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:22.497472 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-af56c666-c8f3-459b-9dae-17fdcd9ea57a None None] MFA Rules not processed for user `4269fd18e2484e29984279dc9873ebb7`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:22.499154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-af56c666-c8f3-459b-9dae-17fdcd9ea57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:22.569048 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29039edc-7d20-4d2c-97ca-5a565b96a80f admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:22.574381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9985d5-5fe0-46a7-81d7-a3ab1f134af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:22.575330 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9985d5-5fe0-46a7-81d7-a3ab1f134af6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:22.575881 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9985d5-5fe0-46a7-81d7-a3ab1f134af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:22.575993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9985d5-5fe0-46a7-81d7-a3ab1f134af6 admin admin] PATH_INFO: `/v3/projects/5d40f14dc6f74873ad5f5d10773f5659` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:22.576267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 993/1998] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:22.577174 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9985d5-5fe0-46a7-81d7-a3ab1f134af6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d40f14dc6f74873ad5f5d10773f5659)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:22.577311 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56c666-c8f3-459b-9dae-17fdcd9ea57a None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:22.579737 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9985d5-5fe0-46a7-81d7-a3ab1f134af6 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:22.590177 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56c666-c8f3-459b-9dae-17fdcd9ea57a None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:22.596425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648def7-3c3d-4979-b42b-df12cc9a465a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:22.597247 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648def7-3c3d-4979-b42b-df12cc9a465a admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:22.597536 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648def7-3c3d-4979-b42b-df12cc9a465a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:22.598145 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648def7-3c3d-4979-b42b-df12cc9a465a admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:22.598834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648def7-3c3d-4979-b42b-df12cc9a465a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:22.600736 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648def7-3c3d-4979-b42b-df12cc9a465a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:22.630426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ac9985d5-5fe0-46a7-81d7-a3ab1f134af6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d40f14dc6f74873ad5f5d10773f5659', 'request_id': 'req-ac9985d5-5fe0-46a7-81d7-a3ab1f134af6'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:22.636198 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ac9985d5-5fe0-46a7-81d7-a3ab1f134af6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d40f14dc6f74873ad5f5d10773f5659', 'request_id': 'req-ac9985d5-5fe0-46a7-81d7-a3ab1f134af6'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:22.641150 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 994/1999] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:22 2022] POST /identity/v3/projects => generated 389 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:22.644300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ac9985d5-5fe0-46a7-81d7-a3ab1f134af6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d40f14dc6f74873ad5f5d10773f5659', 'request_id': 'req-ac9985d5-5fe0-46a7-81d7-a3ab1f134af6'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:22.647488 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648def7-3c3d-4979-b42b-df12cc9a465a admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:22.660531 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ac9985d5-5fe0-46a7-81d7-a3ab1f134af6 admin admin] The token cache is being invalidate because project 5d40f14dc6f74873ad5f5d10773f5659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:22.660829 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ac9985d5-5fe0-46a7-81d7-a3ab1f134af6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:22.661547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ac9985d5-5fe0-46a7-81d7-a3ab1f134af6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:22.663952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1006/2000] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:22 2022] DELETE /identity/v3/projects/5d40f14dc6f74873ad5f5d10773f5659 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:03:22.667841 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9985d5-5fe0-46a7-81d7-a3ab1f134af6 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:22.669256 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ac9985d5-5fe0-46a7-81d7-a3ab1f134af6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:22.714257 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648def7-3c3d-4979-b42b-df12cc9a465a admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:22.720089 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4afca-f2db-48ec-ab81-5d43cef748d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:22.720751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4afca-f2db-48ec-ab81-5d43cef748d7 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:22.721031 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4afca-f2db-48ec-ab81-5d43cef748d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:22.721301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4afca-f2db-48ec-ab81-5d43cef748d7 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:22.721981 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4afca-f2db-48ec-ab81-5d43cef748d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:22.723917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4afca-f2db-48ec-ab81-5d43cef748d7 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:22.736816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9985d5-5fe0-46a7-81d7-a3ab1f134af6 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:22.742393 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a4b7a-f95e-4fe1-81be-1a90b7bd96ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:22.743268 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a4b7a-f95e-4fe1-81be-1a90b7bd96ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:22.743550 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a4b7a-f95e-4fe1-81be-1a90b7bd96ba admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:22.743826 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a4b7a-f95e-4fe1-81be-1a90b7bd96ba admin admin] PATH_INFO: `/v3/users/d36350dfa14544e2ae1225c8adb90b17` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:22.744173 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 995/2001] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:22 2022] POST /identity/v3/users => generated 301 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:22.745029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a4b7a-f95e-4fe1-81be-1a90b7bd96ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=d36350dfa14544e2ae1225c8adb90b17)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:22.747059 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a4b7a-f95e-4fe1-81be-1a90b7bd96ba admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:22.747982 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4afca-f2db-48ec-ab81-5d43cef748d7 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:22.749592 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d8c4afca-f2db-48ec-ab81-5d43cef748d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:22.761813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4afca-f2db-48ec-ab81-5d43cef748d7 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:22.767025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84eb2cb-b1bd-490f-9348-0553734b07a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:22.767674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84eb2cb-b1bd-490f-9348-0553734b07a1 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:22.767961 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84eb2cb-b1bd-490f-9348-0553734b07a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:22.768259 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84eb2cb-b1bd-490f-9348-0553734b07a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:22.768975 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84eb2cb-b1bd-490f-9348-0553734b07a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:22.770579 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84eb2cb-b1bd-490f-9348-0553734b07a1 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:22.780999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 996/2002] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:22.785131 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84eb2cb-b1bd-490f-9348-0553734b07a1 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:22.794373 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84eb2cb-b1bd-490f-9348-0553734b07a1 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:22.794982 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-013a4b7a-f95e-4fe1-81be-1a90b7bd96ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd36350dfa14544e2ae1225c8adb90b17'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:22.799002 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca0f83-79e3-426e-9ca5-d595f2b58b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:22.799846 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca0f83-79e3-426e-9ca5-d595f2b58b67 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:22.800124 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca0f83-79e3-426e-9ca5-d595f2b58b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:22.800396 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca0f83-79e3-426e-9ca5-d595f2b58b67 admin admin] PATH_INFO: `/v3/projects/65fed8e596ad421baefa92f79157aeaf/users/59c231feb9b14f179aac7afdbb6b4596/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:22.802486 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-013a4b7a-f95e-4fe1-81be-1a90b7bd96ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd36350dfa14544e2ae1225c8adb90b17'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:22.811411 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-013a4b7a-f95e-4fe1-81be-1a90b7bd96ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd36350dfa14544e2ae1225c8adb90b17'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:22.812170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-013a4b7a-f95e-4fe1-81be-1a90b7bd96ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd36350dfa14544e2ae1225c8adb90b17'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:22.817912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca0f83-79e3-426e-9ca5-d595f2b58b67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65fed8e596ad421baefa92f79157aeaf, user_id=59c231feb9b14f179aac7afdbb6b4596, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:22.820600 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca0f83-79e3-426e-9ca5-d595f2b58b67 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:22.824817 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-013a4b7a-f95e-4fe1-81be-1a90b7bd96ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd36350dfa14544e2ae1225c8adb90b17'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:22.831753 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1007/2003] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:22 2022] DELETE /identity/v3/users/d36350dfa14544e2ae1225c8adb90b17 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:03:22.833629 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 997/2004] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:22 2022] PUT /identity/v3/projects/65fed8e596ad421baefa92f79157aeaf/users/59c231feb9b14f179aac7afdbb6b4596/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:03:22.837368 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a4b7a-f95e-4fe1-81be-1a90b7bd96ba admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:22.838976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-013a4b7a-f95e-4fe1-81be-1a90b7bd96ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:22.907820 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a4b7a-f95e-4fe1-81be-1a90b7bd96ba admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:22.912519 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4c709-99f6-4135-8c47-ca74cadd5188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:22.913216 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4c709-99f6-4135-8c47-ca74cadd5188 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:22.913499 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4c709-99f6-4135-8c47-ca74cadd5188 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:22.913773 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4c709-99f6-4135-8c47-ca74cadd5188 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:22.914657 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4c709-99f6-4135-8c47-ca74cadd5188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:22.916249 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4c709-99f6-4135-8c47-ca74cadd5188 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:22.929014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1008/2005] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:22.932995 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca0f83-79e3-426e-9ca5-d595f2b58b67 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:23.009837 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca0f83-79e3-426e-9ca5-d595f2b58b67 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:23.014801 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f3e6c-681c-4b5b-b354-fdb34f32672e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:23.016385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f3e6c-681c-4b5b-b354-fdb34f32672e admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:23.016664 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f3e6c-681c-4b5b-b354-fdb34f32672e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:23.016936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f3e6c-681c-4b5b-b354-fdb34f32672e admin admin] PATH_INFO: `/v3/projects/65fed8e596ad421baefa92f79157aeaf/users/59c231feb9b14f179aac7afdbb6b4596/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:23.018963 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f3e6c-681c-4b5b-b354-fdb34f32672e admin admin] RBAC: Authorizing `identity:create_grant(project_id=65fed8e596ad421baefa92f79157aeaf, user_id=59c231feb9b14f179aac7afdbb6b4596, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:23.021978 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f3e6c-681c-4b5b-b354-fdb34f32672e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:23.038196 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 998/2006] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:22 2022] PUT /identity/v3/projects/65fed8e596ad421baefa92f79157aeaf/users/59c231feb9b14f179aac7afdbb6b4596/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:23.050101 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6ea3c-844c-4a0c-a884-3502efb83b32 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:23.050101 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6ea3c-844c-4a0c-a884-3502efb83b32 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:23.050101 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6ea3c-844c-4a0c-a884-3502efb83b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:23.075825 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-57e6ea3c-844c-4a0c-a884-3502efb83b32 None None] MFA Rules not processed for user `59c231feb9b14f179aac7afdbb6b4596`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:23.077130 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-57e6ea3c-844c-4a0c-a884-3502efb83b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:23.149427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1009/2007] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:23 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:23.175481 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f3e6c-681c-4b5b-b354-fdb34f32672e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:23.177460 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-984f3e6c-681c-4b5b-b354-fdb34f32672e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:23.184277 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6ea3c-844c-4a0c-a884-3502efb83b32 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:23.185544 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-57e6ea3c-844c-4a0c-a884-3502efb83b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:23.239605 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f3e6c-681c-4b5b-b354-fdb34f32672e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:23.243436 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4598cc9-f039-4046-9176-737290b8de48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:23.245103 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4598cc9-f039-4046-9176-737290b8de48 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:23.245103 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4598cc9-f039-4046-9176-737290b8de48 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:23.245103 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4598cc9-f039-4046-9176-737290b8de48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:23.255579 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4598cc9-f039-4046-9176-737290b8de48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:23.257309 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4598cc9-f039-4046-9176-737290b8de48 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:23.259855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6ea3c-844c-4a0c-a884-3502efb83b32 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:23.264728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14885242-b9e9-417a-a2a6-c60288ae365a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:23.265701 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14885242-b9e9-417a-a2a6-c60288ae365a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:23.266048 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14885242-b9e9-417a-a2a6-c60288ae365a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:23.266417 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14885242-b9e9-417a-a2a6-c60288ae365a admin admin] PATH_INFO: `/v3/projects/8dc026a98fd841718f4be06f6e69d016` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:23.268351 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14885242-b9e9-417a-a2a6-c60288ae365a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dc026a98fd841718f4be06f6e69d016)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:23.269086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 999/2008] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:23 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 23:03:23.270784 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14885242-b9e9-417a-a2a6-c60288ae365a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:23.324119 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-14885242-b9e9-417a-a2a6-c60288ae365a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dc026a98fd841718f4be06f6e69d016', 'request_id': 'req-14885242-b9e9-417a-a2a6-c60288ae365a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:23.331146 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-14885242-b9e9-417a-a2a6-c60288ae365a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dc026a98fd841718f4be06f6e69d016', 'request_id': 'req-14885242-b9e9-417a-a2a6-c60288ae365a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:23.337515 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-14885242-b9e9-417a-a2a6-c60288ae365a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dc026a98fd841718f4be06f6e69d016', 'request_id': 'req-14885242-b9e9-417a-a2a6-c60288ae365a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:23.353702 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-14885242-b9e9-417a-a2a6-c60288ae365a admin admin] The token cache is being invalidate because project 8dc026a98fd841718f4be06f6e69d016 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:23.353702 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-14885242-b9e9-417a-a2a6-c60288ae365a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:23.353702 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-14885242-b9e9-417a-a2a6-c60288ae365a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:23.354180 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1010/2009] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:23 2022] DELETE /identity/v3/projects/8dc026a98fd841718f4be06f6e69d016 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:23.361132 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d30d7-e287-4346-bbc5-c53d354779ab None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:23.361524 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d30d7-e287-4346-bbc5-c53d354779ab None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:23.361865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d30d7-e287-4346-bbc5-c53d354779ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:23.381451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d34d30d7-e287-4346-bbc5-c53d354779ab None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:23.382730 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d34d30d7-e287-4346-bbc5-c53d354779ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:23.400830 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14885242-b9e9-417a-a2a6-c60288ae365a admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:23.402259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-14885242-b9e9-417a-a2a6-c60288ae365a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:23.442285 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1000/2010] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:23.450092 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d659e6-135e-4787-8575-6ae80df978ad None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:23.450545 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d659e6-135e-4787-8575-6ae80df978ad None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:23.450545 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d659e6-135e-4787-8575-6ae80df978ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:23.469152 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14885242-b9e9-417a-a2a6-c60288ae365a admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:23.470331 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-77d659e6-135e-4787-8575-6ae80df978ad None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:23.471578 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-77d659e6-135e-4787-8575-6ae80df978ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:23.473861 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe0eec-1660-4b32-8db1-9f0bb52bf974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:23.475389 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe0eec-1660-4b32-8db1-9f0bb52bf974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:23.475683 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe0eec-1660-4b32-8db1-9f0bb52bf974 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:23.475991 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe0eec-1660-4b32-8db1-9f0bb52bf974 admin admin] PATH_INFO: `/v3/users/4269fd18e2484e29984279dc9873ebb7` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:23.477164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe0eec-1660-4b32-8db1-9f0bb52bf974 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4269fd18e2484e29984279dc9873ebb7)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:23.479162 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe0eec-1660-4b32-8db1-9f0bb52bf974 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:23.479473 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1001/2011] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:03:23.483880 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d659e6-135e-4787-8575-6ae80df978ad None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:23.492823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d659e6-135e-4787-8575-6ae80df978ad None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:23.497958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23480369-e3e8-4815-b007-08a48d47c836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:23.497958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23480369-e3e8-4815-b007-08a48d47c836 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:23.498321 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23480369-e3e8-4815-b007-08a48d47c836 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:23.498591 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23480369-e3e8-4815-b007-08a48d47c836 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:23.499304 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23480369-e3e8-4815-b007-08a48d47c836 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:03:23.499611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23480369-e3e8-4815-b007-08a48d47c836 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:23.501730 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23480369-e3e8-4815-b007-08a48d47c836 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:23.517473 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1002/2012] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 23:03:23.521734 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23480369-e3e8-4815-b007-08a48d47c836 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:23.529436 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2dfe0eec-1660-4b32-8db1-9f0bb52bf974 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4269fd18e2484e29984279dc9873ebb7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:23.531443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23480369-e3e8-4815-b007-08a48d47c836 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:23.536036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0535f2-7429-4448-bbb8-23b683594cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:23.536405 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2dfe0eec-1660-4b32-8db1-9f0bb52bf974 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4269fd18e2484e29984279dc9873ebb7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:23.536671 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0535f2-7429-4448-bbb8-23b683594cf1 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:23.536952 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0535f2-7429-4448-bbb8-23b683594cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:23.537220 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0535f2-7429-4448-bbb8-23b683594cf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:23.537943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0535f2-7429-4448-bbb8-23b683594cf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:23.539852 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0535f2-7429-4448-bbb8-23b683594cf1 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:23.544561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2dfe0eec-1660-4b32-8db1-9f0bb52bf974 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4269fd18e2484e29984279dc9873ebb7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:23.545209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2dfe0eec-1660-4b32-8db1-9f0bb52bf974 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4269fd18e2484e29984279dc9873ebb7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:23.557021 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2dfe0eec-1660-4b32-8db1-9f0bb52bf974 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4269fd18e2484e29984279dc9873ebb7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:23.564109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1011/2013] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:23 2022] DELETE /identity/v3/users/4269fd18e2484e29984279dc9873ebb7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:03:23.568321 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1003/2014] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:23 2022] POST /identity/v3/projects => generated 367 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:03:23.568711 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87bc5fd-b9f6-43d6-a2a3-232e202b49b0 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:23.568999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87bc5fd-b9f6-43d6-a2a3-232e202b49b0 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:23.569216 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0535f2-7429-4448-bbb8-23b683594cf1 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:23.569308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87bc5fd-b9f6-43d6-a2a3-232e202b49b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:23.570819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3c0535f2-7429-4448-bbb8-23b683594cf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:23.586922 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f87bc5fd-b9f6-43d6-a2a3-232e202b49b0 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:23.588155 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f87bc5fd-b9f6-43d6-a2a3-232e202b49b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:23.635480 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0535f2-7429-4448-bbb8-23b683594cf1 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:23.640180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931954b8-8737-4440-8eb8-58882c3aa5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:23.641491 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931954b8-8737-4440-8eb8-58882c3aa5e5 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:23.641763 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931954b8-8737-4440-8eb8-58882c3aa5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:23.642034 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931954b8-8737-4440-8eb8-58882c3aa5e5 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:23.642750 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931954b8-8737-4440-8eb8-58882c3aa5e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:23.644695 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931954b8-8737-4440-8eb8-58882c3aa5e5 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:23.649023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1012/2015] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:03:23.659387 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87bc5fd-b9f6-43d6-a2a3-232e202b49b0 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:23.661149 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f87bc5fd-b9f6-43d6-a2a3-232e202b49b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:23.665660 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1004/2016] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:23 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:23.669554 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931954b8-8737-4440-8eb8-58882c3aa5e5 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:23.679129 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931954b8-8737-4440-8eb8-58882c3aa5e5 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:23.683844 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9bd92-7440-4ca0-b81b-bd9008ca2719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:23.684489 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9bd92-7440-4ca0-b81b-bd9008ca2719 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:23.684772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9bd92-7440-4ca0-b81b-bd9008ca2719 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:23.685050 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9bd92-7440-4ca0-b81b-bd9008ca2719 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:23.685760 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9bd92-7440-4ca0-b81b-bd9008ca2719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:23.687884 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9bd92-7440-4ca0-b81b-bd9008ca2719 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:23.697409 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1005/2017] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:23.701428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9bd92-7440-4ca0-b81b-bd9008ca2719 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:23.711475 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9bd92-7440-4ca0-b81b-bd9008ca2719 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:23.716394 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9519adc-bcec-4a08-aeeb-e6e90b309c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:23.717139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9519adc-bcec-4a08-aeeb-e6e90b309c41 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:23.717499 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9519adc-bcec-4a08-aeeb-e6e90b309c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:23.717780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9519adc-bcec-4a08-aeeb-e6e90b309c41 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:23.718536 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9519adc-bcec-4a08-aeeb-e6e90b309c41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:23.718948 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87bc5fd-b9f6-43d6-a2a3-232e202b49b0 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:23.720125 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9519adc-bcec-4a08-aeeb-e6e90b309c41 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:23.722589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89491cd4-ddb1-41db-8acf-eef25a06ca88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:23.723224 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89491cd4-ddb1-41db-8acf-eef25a06ca88 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:23.723506 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89491cd4-ddb1-41db-8acf-eef25a06ca88 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:23.723781 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89491cd4-ddb1-41db-8acf-eef25a06ca88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:23.733183 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1006/2018] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:23.733377 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89491cd4-ddb1-41db-8acf-eef25a06ca88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:23.734396 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9519adc-bcec-4a08-aeeb-e6e90b309c41 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:23.734659 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89491cd4-ddb1-41db-8acf-eef25a06ca88 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:23.743722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9519adc-bcec-4a08-aeeb-e6e90b309c41 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:23.744362 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1013/2019] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:23.748758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d638326-630e-41ea-b71f-5dcfebdff089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:23.749693 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d638326-630e-41ea-b71f-5dcfebdff089 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:23.750580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d638326-630e-41ea-b71f-5dcfebdff089 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:23.750933 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d638326-630e-41ea-b71f-5dcfebdff089 admin admin] PATH_INFO: `/v3/projects/d73ac74cb24d432ba9e5496e7c0e291e/users/ed3226d78e4248839447d36e4694d997/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:23.766653 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d638326-630e-41ea-b71f-5dcfebdff089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d73ac74cb24d432ba9e5496e7c0e291e, user_id=ed3226d78e4248839447d36e4694d997, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:23.769373 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d638326-630e-41ea-b71f-5dcfebdff089 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:23.781888 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1007/2020] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:23 2022] PUT /identity/v3/projects/d73ac74cb24d432ba9e5496e7c0e291e/users/ed3226d78e4248839447d36e4694d997/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:23.793242 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5aab4-f92f-4658-b927-e3afb10bf6fe None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:23.793579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5aab4-f92f-4658-b927-e3afb10bf6fe None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:23.793862 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5aab4-f92f-4658-b927-e3afb10bf6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:23.821949 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5de5aab4-f92f-4658-b927-e3afb10bf6fe None None] MFA Rules not processed for user `ed3226d78e4248839447d36e4694d997`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:23.823267 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5de5aab4-f92f-4658-b927-e3afb10bf6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:23.886714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1014/2021] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:23 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:23.899020 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d638326-630e-41ea-b71f-5dcfebdff089 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:23.900807 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8d638326-630e-41ea-b71f-5dcfebdff089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:23.957057 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d638326-630e-41ea-b71f-5dcfebdff089 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:23.960749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe454c4-2f72-4b5b-9299-365f53234cff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81caf0558f2449cbbc8f4cd9446f363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:23.961399 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe454c4-2f72-4b5b-9299-365f53234cff service glance] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:23.961682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe454c4-2f72-4b5b-9299-365f53234cff service glance] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:23.961956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe454c4-2f72-4b5b-9299-365f53234cff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:23.970483 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe454c4-2f72-4b5b-9299-365f53234cff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:23.971907 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe454c4-2f72-4b5b-9299-365f53234cff service glance] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:23.981089 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1008/2022] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:23 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:24.011446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de5aab4-f92f-4658-b927-e3afb10bf6fe None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:24.012915 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5de5aab4-f92f-4658-b927-e3afb10bf6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:24.087325 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de5aab4-f92f-4658-b927-e3afb10bf6fe None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:24.094943 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b580a13-6198-45b7-aa81-be222ca599cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:24.095971 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b580a13-6198-45b7-aa81-be222ca599cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:24.096495 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b580a13-6198-45b7-aa81-be222ca599cd admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:24.096897 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b580a13-6198-45b7-aa81-be222ca599cd admin admin] PATH_INFO: `/v3/projects/2f993851a0b246e5b33460244c3bd355` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:24.098537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b580a13-6198-45b7-aa81-be222ca599cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f993851a0b246e5b33460244c3bd355)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:24.100553 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b580a13-6198-45b7-aa81-be222ca599cd admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:24.152032 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0b580a13-6198-45b7-aa81-be222ca599cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f993851a0b246e5b33460244c3bd355', 'request_id': 'req-0b580a13-6198-45b7-aa81-be222ca599cd'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:24.157564 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0b580a13-6198-45b7-aa81-be222ca599cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f993851a0b246e5b33460244c3bd355', 'request_id': 'req-0b580a13-6198-45b7-aa81-be222ca599cd'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:24.163775 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0b580a13-6198-45b7-aa81-be222ca599cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f993851a0b246e5b33460244c3bd355', 'request_id': 'req-0b580a13-6198-45b7-aa81-be222ca599cd'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:24.179840 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0b580a13-6198-45b7-aa81-be222ca599cd admin admin] The token cache is being invalidate because project 2f993851a0b246e5b33460244c3bd355 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:24.180140 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0b580a13-6198-45b7-aa81-be222ca599cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:24.180960 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0b580a13-6198-45b7-aa81-be222ca599cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:24.182810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1015/2023] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:24 2022] DELETE /identity/v3/projects/2f993851a0b246e5b33460244c3bd355 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:24.187035 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe454c4-2f72-4b5b-9299-365f53234cff service glance] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:24.202830 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe454c4-2f72-4b5b-9299-365f53234cff service glance] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:24.260375 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89070794-d544-418f-8079-44f5a3bb9be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:24.261301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89070794-d544-418f-8079-44f5a3bb9be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:24.261588 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89070794-d544-418f-8079-44f5a3bb9be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:24.261863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89070794-d544-418f-8079-44f5a3bb9be6 admin admin] PATH_INFO: `/v3/users/59c231feb9b14f179aac7afdbb6b4596` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:24.263132 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89070794-d544-418f-8079-44f5a3bb9be6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59c231feb9b14f179aac7afdbb6b4596)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:24.265674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89070794-d544-418f-8079-44f5a3bb9be6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:24.342404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-89070794-d544-418f-8079-44f5a3bb9be6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59c231feb9b14f179aac7afdbb6b4596'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:24.356806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-89070794-d544-418f-8079-44f5a3bb9be6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59c231feb9b14f179aac7afdbb6b4596'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:24.357913 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-89070794-d544-418f-8079-44f5a3bb9be6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59c231feb9b14f179aac7afdbb6b4596'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:24.368906 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-89070794-d544-418f-8079-44f5a3bb9be6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59c231feb9b14f179aac7afdbb6b4596'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:24.369800 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-89070794-d544-418f-8079-44f5a3bb9be6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59c231feb9b14f179aac7afdbb6b4596'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:24.396341 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1009/2024] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:24 2022] DELETE /identity/v3/users/59c231feb9b14f179aac7afdbb6b4596 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:25.014573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b580a13-6198-45b7-aa81-be222ca599cd admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:25.017487 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0b580a13-6198-45b7-aa81-be222ca599cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:25.031689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b580a13-6198-45b7-aa81-be222ca599cd admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:25.103132 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fccac8f-3986-4f68-ba88-61946b3590ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:25.103999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fccac8f-3986-4f68-ba88-61946b3590ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:25.104561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fccac8f-3986-4f68-ba88-61946b3590ab admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:25.104561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fccac8f-3986-4f68-ba88-61946b3590ab admin admin] PATH_INFO: `/v3/projects/65fed8e596ad421baefa92f79157aeaf` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:25.105722 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fccac8f-3986-4f68-ba88-61946b3590ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=65fed8e596ad421baefa92f79157aeaf)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:25.107934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fccac8f-3986-4f68-ba88-61946b3590ab admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:25.161232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4fccac8f-3986-4f68-ba88-61946b3590ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65fed8e596ad421baefa92f79157aeaf', 'request_id': 'req-4fccac8f-3986-4f68-ba88-61946b3590ab'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:25.167442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4fccac8f-3986-4f68-ba88-61946b3590ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65fed8e596ad421baefa92f79157aeaf', 'request_id': 'req-4fccac8f-3986-4f68-ba88-61946b3590ab'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:25.174764 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4fccac8f-3986-4f68-ba88-61946b3590ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65fed8e596ad421baefa92f79157aeaf', 'request_id': 'req-4fccac8f-3986-4f68-ba88-61946b3590ab'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:25.187930 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4fccac8f-3986-4f68-ba88-61946b3590ab admin admin] The token cache is being invalidate because project 65fed8e596ad421baefa92f79157aeaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:25.188219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4fccac8f-3986-4f68-ba88-61946b3590ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:25.188890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4fccac8f-3986-4f68-ba88-61946b3590ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:25.190525 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1016/2025] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:25 2022] DELETE /identity/v3/projects/65fed8e596ad421baefa92f79157aeaf => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:03:25.195807 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0244a-7818-47fe-88d9-10d1dd67136f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:25.196412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0244a-7818-47fe-88d9-10d1dd67136f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:25.196412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0244a-7818-47fe-88d9-10d1dd67136f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:25.215871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ebc0244a-7818-47fe-88d9-10d1dd67136f None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:25.217161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ebc0244a-7818-47fe-88d9-10d1dd67136f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:25.283395 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1010/2026] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:25.291631 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94abbe40-4254-4544-bf94-53166135df1d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:25.291914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94abbe40-4254-4544-bf94-53166135df1d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:25.292216 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94abbe40-4254-4544-bf94-53166135df1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:25.314030 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-94abbe40-4254-4544-bf94-53166135df1d None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:25.315535 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-94abbe40-4254-4544-bf94-53166135df1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:25.324231 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1017/2027] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:25.329063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0244a-7818-47fe-88d9-10d1dd67136f None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:25.339269 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0244a-7818-47fe-88d9-10d1dd67136f None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:25.345914 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c55614-145b-407c-8bb4-2e06183ee1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:25.346699 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c55614-145b-407c-8bb4-2e06183ee1cd admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:25.347036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c55614-145b-407c-8bb4-2e06183ee1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:25.347397 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c55614-145b-407c-8bb4-2e06183ee1cd admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:25.348184 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c55614-145b-407c-8bb4-2e06183ee1cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:03:25.348442 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c55614-145b-407c-8bb4-2e06183ee1cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:25.350291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c55614-145b-407c-8bb4-2e06183ee1cd admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:25.370832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94abbe40-4254-4544-bf94-53166135df1d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:25.374345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1011/2028] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:03:25.381242 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94abbe40-4254-4544-bf94-53166135df1d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:25.387615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161a12e-1c61-494a-a78a-2ce17b24034c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:25.387615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161a12e-1c61-494a-a78a-2ce17b24034c admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:25.398523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161a12e-1c61-494a-a78a-2ce17b24034c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:25.399146 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161a12e-1c61-494a-a78a-2ce17b24034c admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:25.400049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161a12e-1c61-494a-a78a-2ce17b24034c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:25.402417 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161a12e-1c61-494a-a78a-2ce17b24034c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:25.450154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1018/2029] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:25 2022] POST /identity/v3/projects => generated 383 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:03:25.452746 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c55614-145b-407c-8bb4-2e06183ee1cd admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:25.464958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c55614-145b-407c-8bb4-2e06183ee1cd admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:25.529302 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8656a6b-2371-47c7-8b24-d218f881f12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:25.529981 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8656a6b-2371-47c7-8b24-d218f881f12d admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:25.530304 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8656a6b-2371-47c7-8b24-d218f881f12d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:25.530582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8656a6b-2371-47c7-8b24-d218f881f12d admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:25.531872 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8656a6b-2371-47c7-8b24-d218f881f12d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:25.534029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8656a6b-2371-47c7-8b24-d218f881f12d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:25.559873 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1012/2030] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:25 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:03:25.561106 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161a12e-1c61-494a-a78a-2ce17b24034c admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:25.572147 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161a12e-1c61-494a-a78a-2ce17b24034c admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:25.576854 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cda3b2-3e9d-435e-bff4-738f9c2fc93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:25.577572 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cda3b2-3e9d-435e-bff4-738f9c2fc93e admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:25.578385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cda3b2-3e9d-435e-bff4-738f9c2fc93e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:25.578669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cda3b2-3e9d-435e-bff4-738f9c2fc93e admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:25.579393 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cda3b2-3e9d-435e-bff4-738f9c2fc93e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:25.582156 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cda3b2-3e9d-435e-bff4-738f9c2fc93e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:25.591761 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1019/2031] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:25.596840 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8656a6b-2371-47c7-8b24-d218f881f12d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:25.608954 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8656a6b-2371-47c7-8b24-d218f881f12d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:25.614266 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f8eb2-bc5c-4413-800c-69a4a4ac29b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:25.614905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f8eb2-bc5c-4413-800c-69a4a4ac29b4 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:25.615296 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f8eb2-bc5c-4413-800c-69a4a4ac29b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:25.616163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f8eb2-bc5c-4413-800c-69a4a4ac29b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:25.616287 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f8eb2-bc5c-4413-800c-69a4a4ac29b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:25.618179 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f8eb2-bc5c-4413-800c-69a4a4ac29b4 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:25.633196 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1013/2032] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:25.633637 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cda3b2-3e9d-435e-bff4-738f9c2fc93e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:25.648216 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cda3b2-3e9d-435e-bff4-738f9c2fc93e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:25.653177 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f32bc2-134f-4640-b6e4-435319932c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:25.654180 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f32bc2-134f-4640-b6e4-435319932c0e admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:25.654808 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f32bc2-134f-4640-b6e4-435319932c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:25.654808 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f32bc2-134f-4640-b6e4-435319932c0e admin admin] PATH_INFO: `/v3/projects/57575d51d9c94be1b8295c9020265729/users/ed99b1ccc564474d882865431cebe6cb/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:25.671697 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f32bc2-134f-4640-b6e4-435319932c0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=57575d51d9c94be1b8295c9020265729, user_id=ed99b1ccc564474d882865431cebe6cb, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:25.676142 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f32bc2-134f-4640-b6e4-435319932c0e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:25.690626 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1020/2033] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:25 2022] PUT /identity/v3/projects/57575d51d9c94be1b8295c9020265729/users/ed99b1ccc564474d882865431cebe6cb/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:03:25.701885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77395dac-cbdc-4e1f-af6d-05d72e5466dd None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:25.702191 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77395dac-cbdc-4e1f-af6d-05d72e5466dd None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:25.702469 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77395dac-cbdc-4e1f-af6d-05d72e5466dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:25.749550 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-77395dac-cbdc-4e1f-af6d-05d72e5466dd None None] MFA Rules not processed for user `ed99b1ccc564474d882865431cebe6cb`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:25.751405 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-77395dac-cbdc-4e1f-af6d-05d72e5466dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:25.829085 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1014/2034] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:03:25.833767 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f32bc2-134f-4640-b6e4-435319932c0e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:25.912073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f32bc2-134f-4640-b6e4-435319932c0e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:25.916905 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4fcd06-f39c-4356-bab7-6e969ae36a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:25.917658 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fcd06-f39c-4356-bab7-6e969ae36a26 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:25.918010 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fcd06-f39c-4356-bab7-6e969ae36a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:25.918383 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fcd06-f39c-4356-bab7-6e969ae36a26 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:25.919144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4fcd06-f39c-4356-bab7-6e969ae36a26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:25.921107 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4fcd06-f39c-4356-bab7-6e969ae36a26 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:25.958166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1021/2035] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:25 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:25.969201 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77395dac-cbdc-4e1f-af6d-05d72e5466dd None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:25.983897 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77395dac-cbdc-4e1f-af6d-05d72e5466dd None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:26.050105 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08c2a8-1f8d-4934-967a-fcf1f5023f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:26.051174 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08c2a8-1f8d-4934-967a-fcf1f5023f9a admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:26.051541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08c2a8-1f8d-4934-967a-fcf1f5023f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:26.051926 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08c2a8-1f8d-4934-967a-fcf1f5023f9a admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:26.053009 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b08c2a8-1f8d-4934-967a-fcf1f5023f9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:26.055671 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b08c2a8-1f8d-4934-967a-fcf1f5023f9a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:26.079675 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1015/2036] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:25 2022] POST /identity/v3/users => generated 298 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:26.083624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4fcd06-f39c-4356-bab7-6e969ae36a26 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:26.094835 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4fcd06-f39c-4356-bab7-6e969ae36a26 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:26.099620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c9bb8-a1c3-4aa5-9b86-0f357c253b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:26.100291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c9bb8-a1c3-4aa5-9b86-0f357c253b77 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:26.100571 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c9bb8-a1c3-4aa5-9b86-0f357c253b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:26.100846 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c9bb8-a1c3-4aa5-9b86-0f357c253b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:26.101562 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c9bb8-a1c3-4aa5-9b86-0f357c253b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:26.103183 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c9bb8-a1c3-4aa5-9b86-0f357c253b77 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:26.117363 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1022/2037] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:26.119506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08c2a8-1f8d-4934-967a-fcf1f5023f9a admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:26.129475 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08c2a8-1f8d-4934-967a-fcf1f5023f9a admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:26.134320 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6aaf9-5a2c-42fa-9a21-db41f5f58e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:26.135191 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6aaf9-5a2c-42fa-9a21-db41f5f58e90 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:26.135473 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6aaf9-5a2c-42fa-9a21-db41f5f58e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:26.135762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6aaf9-5a2c-42fa-9a21-db41f5f58e90 admin admin] PATH_INFO: `/v3/projects/8ba926f610924bfda558d6d7b2ca238c/users/33555c27916241228058f1e02451b12d/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:26.150845 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6aaf9-5a2c-42fa-9a21-db41f5f58e90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ba926f610924bfda558d6d7b2ca238c, user_id=33555c27916241228058f1e02451b12d, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:26.153483 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6aaf9-5a2c-42fa-9a21-db41f5f58e90 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:26.166951 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1016/2038] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:26 2022] PUT /identity/v3/projects/8ba926f610924bfda558d6d7b2ca238c/users/33555c27916241228058f1e02451b12d/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:26.170800 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c9bb8-a1c3-4aa5-9b86-0f357c253b77 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:26.180648 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c9bb8-a1c3-4aa5-9b86-0f357c253b77 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:26.240948 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf3c0d-2d6d-4333-8931-750ecbb729ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:26.241618 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf3c0d-2d6d-4333-8931-750ecbb729ff admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:26.241924 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf3c0d-2d6d-4333-8931-750ecbb729ff admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:26.242448 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf3c0d-2d6d-4333-8931-750ecbb729ff admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:26.243201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf3c0d-2d6d-4333-8931-750ecbb729ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:26.244758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf3c0d-2d6d-4333-8931-750ecbb729ff admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:26.258564 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1023/2039] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:26.260060 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6aaf9-5a2c-42fa-9a21-db41f5f58e90 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:26.277109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6aaf9-5a2c-42fa-9a21-db41f5f58e90 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:26.283961 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da62d0b1-0d5c-406e-b40d-0a296f25a6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:26.285341 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62d0b1-0d5c-406e-b40d-0a296f25a6b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:26.285864 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62d0b1-0d5c-406e-b40d-0a296f25a6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:26.286332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62d0b1-0d5c-406e-b40d-0a296f25a6b7 admin admin] PATH_INFO: `/v3/projects/8ba926f610924bfda558d6d7b2ca238c/users/33555c27916241228058f1e02451b12d/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:26.296850 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da62d0b1-0d5c-406e-b40d-0a296f25a6b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ba926f610924bfda558d6d7b2ca238c, user_id=33555c27916241228058f1e02451b12d, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:26.299575 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da62d0b1-0d5c-406e-b40d-0a296f25a6b7 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:26.310286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1017/2040] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:26 2022] PUT /identity/v3/projects/8ba926f610924bfda558d6d7b2ca238c/users/33555c27916241228058f1e02451b12d/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:03:26.321413 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f19150-317d-4ba5-b2bf-fca51588fb29 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:26.321705 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f19150-317d-4ba5-b2bf-fca51588fb29 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:26.321982 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f19150-317d-4ba5-b2bf-fca51588fb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:26.348547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-97f19150-317d-4ba5-b2bf-fca51588fb29 None None] MFA Rules not processed for user `33555c27916241228058f1e02451b12d`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:26.349745 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-97f19150-317d-4ba5-b2bf-fca51588fb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:26.415016 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1024/2041] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:26.426421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da62d0b1-0d5c-406e-b40d-0a296f25a6b7 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:26.428187 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-da62d0b1-0d5c-406e-b40d-0a296f25a6b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:26.491624 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da62d0b1-0d5c-406e-b40d-0a296f25a6b7 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:26.495348 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eec016-6bbd-4992-b4a5-d1f8d45a8a71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:26.496001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eec016-6bbd-4992-b4a5-d1f8d45a8a71 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:26.496285 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eec016-6bbd-4992-b4a5-d1f8d45a8a71 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:26.496565 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eec016-6bbd-4992-b4a5-d1f8d45a8a71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:26.503321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f19150-317d-4ba5-b2bf-fca51588fb29 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:26.505041 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-97f19150-317d-4ba5-b2bf-fca51588fb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:26.507376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eec016-6bbd-4992-b4a5-d1f8d45a8a71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:26.508854 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eec016-6bbd-4992-b4a5-d1f8d45a8a71 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:26.519121 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1018/2042] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:26 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:26.579394 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f19150-317d-4ba5-b2bf-fca51588fb29 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:26.584804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681b423-1919-4fb9-b8fc-cfcb4e953545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:26.585681 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681b423-1919-4fb9-b8fc-cfcb4e953545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:26.585947 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681b423-1919-4fb9-b8fc-cfcb4e953545 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:26.586290 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681b423-1919-4fb9-b8fc-cfcb4e953545 admin admin] PATH_INFO: `/v3/users/2714b23e7c2b4eceb3c9dedcd04fca27` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:26.587887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681b423-1919-4fb9-b8fc-cfcb4e953545 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2714b23e7c2b4eceb3c9dedcd04fca27)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:26.589725 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681b423-1919-4fb9-b8fc-cfcb4e953545 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:26.615689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eec016-6bbd-4992-b4a5-d1f8d45a8a71 service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:26.617144 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a9eec016-6bbd-4992-b4a5-d1f8d45a8a71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:26.631632 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eec016-6bbd-4992-b4a5-d1f8d45a8a71 service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:26.636582 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1a813-a2b1-4eea-9e4b-f35c37954aa1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:26.637403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1a813-a2b1-4eea-9e4b-f35c37954aa1 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:26.637504 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1a813-a2b1-4eea-9e4b-f35c37954aa1 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:26.637766 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1a813-a2b1-4eea-9e4b-f35c37954aa1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:26.665917 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-1681b423-1919-4fb9-b8fc-cfcb4e953545 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2714b23e7c2b4eceb3c9dedcd04fca27'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:26.675601 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-1681b423-1919-4fb9-b8fc-cfcb4e953545 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2714b23e7c2b4eceb3c9dedcd04fca27'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:26.689770 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-1681b423-1919-4fb9-b8fc-cfcb4e953545 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2714b23e7c2b4eceb3c9dedcd04fca27'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:26.690902 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-1681b423-1919-4fb9-b8fc-cfcb4e953545 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2714b23e7c2b4eceb3c9dedcd04fca27'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:26.706861 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-1681b423-1919-4fb9-b8fc-cfcb4e953545 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2714b23e7c2b4eceb3c9dedcd04fca27'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:26.713752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1025/2043] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:26 2022] DELETE /identity/v3/users/2714b23e7c2b4eceb3c9dedcd04fca27 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:03:26.717861 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1a813-a2b1-4eea-9e4b-f35c37954aa1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:26.718101 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78619e-444f-4a6f-a12e-bdccec819ee3 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:26.718931 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78619e-444f-4a6f-a12e-bdccec819ee3 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:26.719241 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78619e-444f-4a6f-a12e-bdccec819ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:26.720353 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1a813-a2b1-4eea-9e4b-f35c37954aa1 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:26.730160 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1019/2044] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 23:03:26.740143 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-7b78619e-444f-4a6f-a12e-bdccec819ee3 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:26.741388 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7b78619e-444f-4a6f-a12e-bdccec819ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:26.819785 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1026/2045] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:26.829849 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1a813-a2b1-4eea-9e4b-f35c37954aa1 service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:26.831352 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-41e1a813-a2b1-4eea-9e4b-f35c37954aa1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:26.896344 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1a813-a2b1-4eea-9e4b-f35c37954aa1 service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:26.948721 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa4d34-c428-44f3-85f6-e3c85e3c6518 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:26.949365 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa4d34-c428-44f3-85f6-e3c85e3c6518 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:26.949641 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa4d34-c428-44f3-85f6-e3c85e3c6518 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:26.949916 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa4d34-c428-44f3-85f6-e3c85e3c6518 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:26.959469 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa4d34-c428-44f3-85f6-e3c85e3c6518 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:26.960890 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa4d34-c428-44f3-85f6-e3c85e3c6518 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:26.970657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1020/2046] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:27.007589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78619e-444f-4a6f-a12e-bdccec819ee3 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:27.009024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7b78619e-444f-4a6f-a12e-bdccec819ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:27.021959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78619e-444f-4a6f-a12e-bdccec819ee3 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:27.026651 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b475983-00c4-4f71-b9ba-197e045eb2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:27.027550 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b475983-00c4-4f71-b9ba-197e045eb2e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:27.027816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b475983-00c4-4f71-b9ba-197e045eb2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:27.028113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b475983-00c4-4f71-b9ba-197e045eb2e9 admin admin] PATH_INFO: `/v3/users/ed99b1ccc564474d882865431cebe6cb` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:27.034237 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b475983-00c4-4f71-b9ba-197e045eb2e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed99b1ccc564474d882865431cebe6cb)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:27.038605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b475983-00c4-4f71-b9ba-197e045eb2e9 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:27.109261 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-1b475983-00c4-4f71-b9ba-197e045eb2e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed99b1ccc564474d882865431cebe6cb'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:27.117384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-1b475983-00c4-4f71-b9ba-197e045eb2e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed99b1ccc564474d882865431cebe6cb'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:27.126726 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-1b475983-00c4-4f71-b9ba-197e045eb2e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed99b1ccc564474d882865431cebe6cb'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:27.127418 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-1b475983-00c4-4f71-b9ba-197e045eb2e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed99b1ccc564474d882865431cebe6cb'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:27.150053 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-1b475983-00c4-4f71-b9ba-197e045eb2e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed99b1ccc564474d882865431cebe6cb'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:27.157928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1027/2047] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:27 2022] DELETE /identity/v3/users/ed99b1ccc564474d882865431cebe6cb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:27.162556 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a5405-992d-4177-9e94-5a93d9b69bc6 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:27.162839 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a5405-992d-4177-9e94-5a93d9b69bc6 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:27.163119 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a5405-992d-4177-9e94-5a93d9b69bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:27.182327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-621a5405-992d-4177-9e94-5a93d9b69bc6 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:27.183609 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-621a5405-992d-4177-9e94-5a93d9b69bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:27.266680 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1021/2048] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:27.287335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b475983-00c4-4f71-b9ba-197e045eb2e9 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:27.288788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1b475983-00c4-4f71-b9ba-197e045eb2e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:27.291093 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a5405-992d-4177-9e94-5a93d9b69bc6 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:27.292676 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-621a5405-992d-4177-9e94-5a93d9b69bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:27.306090 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a5405-992d-4177-9e94-5a93d9b69bc6 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:27.311252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928eee54-24ee-42cc-98ef-749f675fd26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:27.312704 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928eee54-24ee-42cc-98ef-749f675fd26f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:27.312995 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928eee54-24ee-42cc-98ef-749f675fd26f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:27.313268 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928eee54-24ee-42cc-98ef-749f675fd26f admin admin] PATH_INFO: `/v3/projects/4200d3d033234934a42cc5110f10f278` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:27.314520 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928eee54-24ee-42cc-98ef-749f675fd26f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4200d3d033234934a42cc5110f10f278)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:27.316594 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928eee54-24ee-42cc-98ef-749f675fd26f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:27.372317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b475983-00c4-4f71-b9ba-197e045eb2e9 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:27.387469 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-928eee54-24ee-42cc-98ef-749f675fd26f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4200d3d033234934a42cc5110f10f278', 'request_id': 'req-928eee54-24ee-42cc-98ef-749f675fd26f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:27.394318 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-928eee54-24ee-42cc-98ef-749f675fd26f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4200d3d033234934a42cc5110f10f278', 'request_id': 'req-928eee54-24ee-42cc-98ef-749f675fd26f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:27.400798 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-928eee54-24ee-42cc-98ef-749f675fd26f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4200d3d033234934a42cc5110f10f278', 'request_id': 'req-928eee54-24ee-42cc-98ef-749f675fd26f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:27.414876 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-928eee54-24ee-42cc-98ef-749f675fd26f admin admin] The token cache is being invalidate because project 4200d3d033234934a42cc5110f10f278 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:27.415252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-928eee54-24ee-42cc-98ef-749f675fd26f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:27.415958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-928eee54-24ee-42cc-98ef-749f675fd26f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:27.417809 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1022/2049] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:27 2022] DELETE /identity/v3/projects/4200d3d033234934a42cc5110f10f278 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:27.422270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928eee54-24ee-42cc-98ef-749f675fd26f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:27.424581 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-928eee54-24ee-42cc-98ef-749f675fd26f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:27.438714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ccebb5-9744-48bf-ae34-904e9368c1f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:27.439473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ccebb5-9744-48bf-ae34-904e9368c1f0 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:27.439754 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ccebb5-9744-48bf-ae34-904e9368c1f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:27.440027 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ccebb5-9744-48bf-ae34-904e9368c1f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:27.442060 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-81ccebb5-9744-48bf-ae34-904e9368c1f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:27.503541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928eee54-24ee-42cc-98ef-749f675fd26f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:27.508813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b3cd2-15f4-4952-b335-f829f3e3b78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:27.509340 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b3cd2-15f4-4952-b335-f829f3e3b78a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:27.509467 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ccebb5-9744-48bf-ae34-904e9368c1f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:27.509758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b3cd2-15f4-4952-b335-f829f3e3b78a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:27.510035 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b3cd2-15f4-4952-b335-f829f3e3b78a admin admin] PATH_INFO: `/v3/users/94e7ebc9886c44d387f701e2a3e1dab3` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:27.511251 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ccebb5-9744-48bf-ae34-904e9368c1f0 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:27.511357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b3cd2-15f4-4952-b335-f829f3e3b78a admin admin] RBAC: Authorizing `identity:delete_user(user_id=94e7ebc9886c44d387f701e2a3e1dab3)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:27.513241 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b3cd2-15f4-4952-b335-f829f3e3b78a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:27.522022 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1028/2050] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:27.566546 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-583b3cd2-15f4-4952-b335-f829f3e3b78a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94e7ebc9886c44d387f701e2a3e1dab3'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:27.575171 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-583b3cd2-15f4-4952-b335-f829f3e3b78a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94e7ebc9886c44d387f701e2a3e1dab3'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:27.575810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-583b3cd2-15f4-4952-b335-f829f3e3b78a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94e7ebc9886c44d387f701e2a3e1dab3'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:27.583870 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-583b3cd2-15f4-4952-b335-f829f3e3b78a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94e7ebc9886c44d387f701e2a3e1dab3'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:27.584704 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-583b3cd2-15f4-4952-b335-f829f3e3b78a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94e7ebc9886c44d387f701e2a3e1dab3'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:27.605921 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1023/2051] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:27 2022] DELETE /identity/v3/users/94e7ebc9886c44d387f701e2a3e1dab3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:03:27.848610 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ccebb5-9744-48bf-ae34-904e9368c1f0 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:27.850174 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-81ccebb5-9744-48bf-ae34-904e9368c1f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:27.863099 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ccebb5-9744-48bf-ae34-904e9368c1f0 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:27.954247 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435e6af-807b-4e55-a71a-d61249bbc2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:27.957879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435e6af-807b-4e55-a71a-d61249bbc2c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:27.957879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435e6af-807b-4e55-a71a-d61249bbc2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:27.957879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435e6af-807b-4e55-a71a-d61249bbc2c0 admin admin] PATH_INFO: `/v3/projects/57575d51d9c94be1b8295c9020265729` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:27.959241 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b3cd2-15f4-4952-b335-f829f3e3b78a admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:27.959539 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435e6af-807b-4e55-a71a-d61249bbc2c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57575d51d9c94be1b8295c9020265729)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:27.961415 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-583b3cd2-15f4-4952-b335-f829f3e3b78a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:27.961533 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435e6af-807b-4e55-a71a-d61249bbc2c0 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:27.998738 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b3cd2-15f4-4952-b335-f829f3e3b78a admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:28.005089 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4f63d-a034-4596-9182-a65f39eb2b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:28.006130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4f63d-a034-4596-9182-a65f39eb2b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:28.006429 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4f63d-a034-4596-9182-a65f39eb2b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:28.006699 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4f63d-a034-4596-9182-a65f39eb2b61 admin admin] PATH_INFO: `/v3/projects/dc181a40b42a4f51982126b1bcbfebb2` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:28.008054 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4f63d-a034-4596-9182-a65f39eb2b61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc181a40b42a4f51982126b1bcbfebb2)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:28.009947 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4f63d-a034-4596-9182-a65f39eb2b61 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:28.022360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7435e6af-807b-4e55-a71a-d61249bbc2c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57575d51d9c94be1b8295c9020265729', 'request_id': 'req-7435e6af-807b-4e55-a71a-d61249bbc2c0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.028222 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7435e6af-807b-4e55-a71a-d61249bbc2c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57575d51d9c94be1b8295c9020265729', 'request_id': 'req-7435e6af-807b-4e55-a71a-d61249bbc2c0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.034562 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7435e6af-807b-4e55-a71a-d61249bbc2c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57575d51d9c94be1b8295c9020265729', 'request_id': 'req-7435e6af-807b-4e55-a71a-d61249bbc2c0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.047906 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7435e6af-807b-4e55-a71a-d61249bbc2c0 admin admin] The token cache is being invalidate because project 57575d51d9c94be1b8295c9020265729 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:28.048313 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7435e6af-807b-4e55-a71a-d61249bbc2c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.048942 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7435e6af-807b-4e55-a71a-d61249bbc2c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.050581 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1029/2052] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:27 2022] DELETE /identity/v3/projects/57575d51d9c94be1b8295c9020265729 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:28.054353 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435e6af-807b-4e55-a71a-d61249bbc2c0 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:28.055829 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7435e6af-807b-4e55-a71a-d61249bbc2c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:28.061552 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-12d4f63d-a034-4596-9182-a65f39eb2b61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc181a40b42a4f51982126b1bcbfebb2', 'request_id': 'req-12d4f63d-a034-4596-9182-a65f39eb2b61'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.068905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-12d4f63d-a034-4596-9182-a65f39eb2b61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc181a40b42a4f51982126b1bcbfebb2', 'request_id': 'req-12d4f63d-a034-4596-9182-a65f39eb2b61'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.074866 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-12d4f63d-a034-4596-9182-a65f39eb2b61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc181a40b42a4f51982126b1bcbfebb2', 'request_id': 'req-12d4f63d-a034-4596-9182-a65f39eb2b61'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.089076 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-12d4f63d-a034-4596-9182-a65f39eb2b61 admin admin] The token cache is being invalidate because project dc181a40b42a4f51982126b1bcbfebb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:28.089486 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-12d4f63d-a034-4596-9182-a65f39eb2b61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.090146 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-12d4f63d-a034-4596-9182-a65f39eb2b61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.091830 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1024/2053] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:27 2022] DELETE /identity/v3/projects/dc181a40b42a4f51982126b1bcbfebb2 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:28.096358 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70d144-310c-4f03-b1fb-c7ddd5738d85 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:28.096637 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70d144-310c-4f03-b1fb-c7ddd5738d85 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:28.096909 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70d144-310c-4f03-b1fb-c7ddd5738d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:28.114468 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-3f70d144-310c-4f03-b1fb-c7ddd5738d85 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:28.115658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3f70d144-310c-4f03-b1fb-c7ddd5738d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:28.134259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435e6af-807b-4e55-a71a-d61249bbc2c0 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:28.172686 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1025/2054] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:28.180576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0fa55-7258-4f83-b5c3-174e5b410f5f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:28.180940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0fa55-7258-4f83-b5c3-174e5b410f5f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:28.181316 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0fa55-7258-4f83-b5c3-174e5b410f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:28.191337 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27bfc1-dc33-4754-98b3-d7acbb6498b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:28.192237 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27bfc1-dc33-4754-98b3-d7acbb6498b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:28.192544 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27bfc1-dc33-4754-98b3-d7acbb6498b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:28.192787 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27bfc1-dc33-4754-98b3-d7acbb6498b5 admin admin] PATH_INFO: `/v3/users/33555c27916241228058f1e02451b12d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:28.194003 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27bfc1-dc33-4754-98b3-d7acbb6498b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33555c27916241228058f1e02451b12d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:28.195997 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27bfc1-dc33-4754-98b3-d7acbb6498b5 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:28.201475 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-1fc0fa55-7258-4f83-b5c3-174e5b410f5f None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:28.206573 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1fc0fa55-7258-4f83-b5c3-174e5b410f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:28.212949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1026/2055] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:03:28.217545 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0fa55-7258-4f83-b5c3-174e5b410f5f None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:28.227315 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0fa55-7258-4f83-b5c3-174e5b410f5f None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:28.235508 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc136a21-e29d-4efc-be84-d78206e19105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:28.236483 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc136a21-e29d-4efc-be84-d78206e19105 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:28.237092 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc136a21-e29d-4efc-be84-d78206e19105 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:28.237600 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc136a21-e29d-4efc-be84-d78206e19105 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:28.238770 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc136a21-e29d-4efc-be84-d78206e19105 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:03:28.239256 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc136a21-e29d-4efc-be84-d78206e19105 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:28.244152 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc136a21-e29d-4efc-be84-d78206e19105 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:28.247250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ed27bfc1-dc33-4754-98b3-d7acbb6498b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33555c27916241228058f1e02451b12d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.256981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ed27bfc1-dc33-4754-98b3-d7acbb6498b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33555c27916241228058f1e02451b12d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.267554 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1027/2056] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 23:03:28.270448 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ed27bfc1-dc33-4754-98b3-d7acbb6498b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33555c27916241228058f1e02451b12d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.270622 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ed27bfc1-dc33-4754-98b3-d7acbb6498b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33555c27916241228058f1e02451b12d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.271846 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc136a21-e29d-4efc-be84-d78206e19105 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:28.281691 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc136a21-e29d-4efc-be84-d78206e19105 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:28.284657 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ed27bfc1-dc33-4754-98b3-d7acbb6498b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33555c27916241228058f1e02451b12d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.286370 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc9f241-8da4-4153-a0b3-f69aa81d810a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:28.287015 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9f241-8da4-4153-a0b3-f69aa81d810a admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:28.287292 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9f241-8da4-4153-a0b3-f69aa81d810a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:28.287558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9f241-8da4-4153-a0b3-f69aa81d810a admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:28.288243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc9f241-8da4-4153-a0b3-f69aa81d810a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:28.290398 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc9f241-8da4-4153-a0b3-f69aa81d810a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:28.291801 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1030/2057] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:28 2022] DELETE /identity/v3/users/33555c27916241228058f1e02451b12d => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:03:28.324117 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1028/2058] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:28 2022] POST /identity/v3/projects => generated 399 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:03:28.325079 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27bfc1-dc33-4754-98b3-d7acbb6498b5 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:28.326526 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ed27bfc1-dc33-4754-98b3-d7acbb6498b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:28.407785 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27bfc1-dc33-4754-98b3-d7acbb6498b5 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:28.478514 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329ffe7-c6d6-49c1-8bf4-891b4750255f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:28.479904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329ffe7-c6d6-49c1-8bf4-891b4750255f admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:28.480292 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329ffe7-c6d6-49c1-8bf4-891b4750255f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:28.480673 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329ffe7-c6d6-49c1-8bf4-891b4750255f admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:28.481758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c329ffe7-c6d6-49c1-8bf4-891b4750255f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:28.490842 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c329ffe7-c6d6-49c1-8bf4-891b4750255f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:28.503621 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc9f241-8da4-4153-a0b3-f69aa81d810a admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:28.505541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0fc9f241-8da4-4153-a0b3-f69aa81d810a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:28.518158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1031/2059] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:28 2022] POST /identity/v3/users => generated 300 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:03:28.521620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329ffe7-c6d6-49c1-8bf4-891b4750255f admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:28.523330 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc9f241-8da4-4153-a0b3-f69aa81d810a admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:28.529250 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04c1ff-107c-40cd-97b2-bb9d082d364d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:28.531034 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04c1ff-107c-40cd-97b2-bb9d082d364d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:28.531605 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04c1ff-107c-40cd-97b2-bb9d082d364d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:28.532019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04c1ff-107c-40cd-97b2-bb9d082d364d admin admin] PATH_INFO: `/v3/users/ed0a95e385dd4dfb962452465668cc3f` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:28.533533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04c1ff-107c-40cd-97b2-bb9d082d364d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed0a95e385dd4dfb962452465668cc3f)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:28.533899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329ffe7-c6d6-49c1-8bf4-891b4750255f admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:28.536213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04c1ff-107c-40cd-97b2-bb9d082d364d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:28.539195 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880388ce-b55d-4346-9764-0e1771b79530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:28.539865 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880388ce-b55d-4346-9764-0e1771b79530 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:28.540147 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880388ce-b55d-4346-9764-0e1771b79530 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:28.540422 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880388ce-b55d-4346-9764-0e1771b79530 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:28.541143 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880388ce-b55d-4346-9764-0e1771b79530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:28.542859 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880388ce-b55d-4346-9764-0e1771b79530 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:28.564524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1032/2060] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:28.568580 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880388ce-b55d-4346-9764-0e1771b79530 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:28.601398 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880388ce-b55d-4346-9764-0e1771b79530 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:28.608889 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a73a32-ed5c-4b02-97f5-fa31d9647efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:28.610955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a73a32-ed5c-4b02-97f5-fa31d9647efa admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:28.611242 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a73a32-ed5c-4b02-97f5-fa31d9647efa admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:28.611578 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a73a32-ed5c-4b02-97f5-fa31d9647efa admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:28.612324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a73a32-ed5c-4b02-97f5-fa31d9647efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:28.617931 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a73a32-ed5c-4b02-97f5-fa31d9647efa admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:28.620443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5f04c1ff-107c-40cd-97b2-bb9d082d364d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed0a95e385dd4dfb962452465668cc3f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.630439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1033/2061] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:28.633737 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5f04c1ff-107c-40cd-97b2-bb9d082d364d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed0a95e385dd4dfb962452465668cc3f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.634643 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a73a32-ed5c-4b02-97f5-fa31d9647efa admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:28.635274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5f04c1ff-107c-40cd-97b2-bb9d082d364d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed0a95e385dd4dfb962452465668cc3f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.636348 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-57a73a32-ed5c-4b02-97f5-fa31d9647efa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:28.643843 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5f04c1ff-107c-40cd-97b2-bb9d082d364d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed0a95e385dd4dfb962452465668cc3f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.644922 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5f04c1ff-107c-40cd-97b2-bb9d082d364d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed0a95e385dd4dfb962452465668cc3f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.651238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a73a32-ed5c-4b02-97f5-fa31d9647efa admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:28.656516 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a029cd8-2db2-466f-b532-10b60f460845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:28.657508 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a029cd8-2db2-466f-b532-10b60f460845 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:28.657848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a029cd8-2db2-466f-b532-10b60f460845 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:28.664997 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a029cd8-2db2-466f-b532-10b60f460845 admin admin] PATH_INFO: `/v3/projects/377be26fb207447ba6baad14693c76f1/users/77423f25c92243bb8ae21f420cabdf90/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:28.679240 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1029/2062] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:28 2022] DELETE /identity/v3/users/ed0a95e385dd4dfb962452465668cc3f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:03:28.684392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04c1ff-107c-40cd-97b2-bb9d082d364d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:28.685948 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5f04c1ff-107c-40cd-97b2-bb9d082d364d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:28.701535 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a029cd8-2db2-466f-b532-10b60f460845 admin admin] RBAC: Authorizing `identity:create_grant(project_id=377be26fb207447ba6baad14693c76f1, user_id=77423f25c92243bb8ae21f420cabdf90, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:28.705081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a029cd8-2db2-466f-b532-10b60f460845 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:28.735280 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1034/2063] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:28 2022] PUT /identity/v3/projects/377be26fb207447ba6baad14693c76f1/users/77423f25c92243bb8ae21f420cabdf90/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:03:28.735280 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4782a03-d59c-4a5d-a7f4-1e66b0d5088c None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:28.735280 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4782a03-d59c-4a5d-a7f4-1e66b0d5088c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:28.735280 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4782a03-d59c-4a5d-a7f4-1e66b0d5088c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:28.754687 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e4782a03-d59c-4a5d-a7f4-1e66b0d5088c None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:28.756004 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e4782a03-d59c-4a5d-a7f4-1e66b0d5088c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:28.773912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04c1ff-107c-40cd-97b2-bb9d082d364d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:28.829469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1035/2064] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:28.829469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9a3a7-6ec6-4116-9891-8088e06a8058 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:28.829469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9a3a7-6ec6-4116-9891-8088e06a8058 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:28.829469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9a3a7-6ec6-4116-9891-8088e06a8058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:28.846570 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c59dcc-48d8-4b6d-8f3c-3df3ad6a75b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:28.847538 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c59dcc-48d8-4b6d-8f3c-3df3ad6a75b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:28.847814 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c59dcc-48d8-4b6d-8f3c-3df3ad6a75b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:28.848085 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c59dcc-48d8-4b6d-8f3c-3df3ad6a75b9 admin admin] PATH_INFO: `/v3/projects/8ba926f610924bfda558d6d7b2ca238c` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:28.849270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c59dcc-48d8-4b6d-8f3c-3df3ad6a75b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ba926f610924bfda558d6d7b2ca238c)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:28.851265 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c59dcc-48d8-4b6d-8f3c-3df3ad6a75b9 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:28.868715 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-0ac9a3a7-6ec6-4116-9891-8088e06a8058 None None] MFA Rules not processed for user `77423f25c92243bb8ae21f420cabdf90`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:28.870023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0ac9a3a7-6ec6-4116-9891-8088e06a8058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:28.899418 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-80c59dcc-48d8-4b6d-8f3c-3df3ad6a75b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ba926f610924bfda558d6d7b2ca238c', 'request_id': 'req-80c59dcc-48d8-4b6d-8f3c-3df3ad6a75b9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.905830 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-80c59dcc-48d8-4b6d-8f3c-3df3ad6a75b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ba926f610924bfda558d6d7b2ca238c', 'request_id': 'req-80c59dcc-48d8-4b6d-8f3c-3df3ad6a75b9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.911567 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-80c59dcc-48d8-4b6d-8f3c-3df3ad6a75b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ba926f610924bfda558d6d7b2ca238c', 'request_id': 'req-80c59dcc-48d8-4b6d-8f3c-3df3ad6a75b9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.924010 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-80c59dcc-48d8-4b6d-8f3c-3df3ad6a75b9 admin admin] The token cache is being invalidate because project 8ba926f610924bfda558d6d7b2ca238c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:28.924292 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-80c59dcc-48d8-4b6d-8f3c-3df3ad6a75b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.924914 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-80c59dcc-48d8-4b6d-8f3c-3df3ad6a75b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:28.926496 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1030/2065] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:28 2022] DELETE /identity/v3/projects/8ba926f610924bfda558d6d7b2ca238c => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:28.927244 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c59dcc-48d8-4b6d-8f3c-3df3ad6a75b9 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:28.930741 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-80c59dcc-48d8-4b6d-8f3c-3df3ad6a75b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:28.938093 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1036/2066] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:28.938093 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c50b5e-555f-48c0-ba29-f02cc58096fa None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:28.938093 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c50b5e-555f-48c0-ba29-f02cc58096fa None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:28.938093 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c50b5e-555f-48c0-ba29-f02cc58096fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:28.958173 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-64c50b5e-555f-48c0-ba29-f02cc58096fa None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:28.959393 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-64c50b5e-555f-48c0-ba29-f02cc58096fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:28.988829 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c59dcc-48d8-4b6d-8f3c-3df3ad6a75b9 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:28.992799 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d4021-8298-4301-bcd1-593b97436910 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:28.993445 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d4021-8298-4301-bcd1-593b97436910 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:28.993753 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d4021-8298-4301-bcd1-593b97436910 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:28.994029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d4021-8298-4301-bcd1-593b97436910 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:28.995898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b05d4021-8298-4301-bcd1-593b97436910 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:29.020148 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1037/2067] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:29.020858 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c50b5e-555f-48c0-ba29-f02cc58096fa None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:29.027304 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-64c50b5e-555f-48c0-ba29-f02cc58096fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:29.039617 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c50b5e-555f-48c0-ba29-f02cc58096fa None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:29.044763 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742aef4-3b99-453d-ab79-ca4961703fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:29.045734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742aef4-3b99-453d-ab79-ca4961703fb4 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:29.046029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742aef4-3b99-453d-ab79-ca4961703fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:29.046327 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742aef4-3b99-453d-ab79-ca4961703fb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:29.047053 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742aef4-3b99-453d-ab79-ca4961703fb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:29.048926 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742aef4-3b99-453d-ab79-ca4961703fb4 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:29.063563 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d4021-8298-4301-bcd1-593b97436910 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:29.065006 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d4021-8298-4301-bcd1-593b97436910 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:29.075168 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1031/2068] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:29.078721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1038/2069] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:29 2022] POST /identity/v3/projects => generated 401 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:29.082772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742aef4-3b99-453d-ab79-ca4961703fb4 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:29.084043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd341f8b-eb15-4664-bc71-7b10fa0be10c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:29.084334 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd341f8b-eb15-4664-bc71-7b10fa0be10c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:29.084610 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd341f8b-eb15-4664-bc71-7b10fa0be10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:29.114529 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-bd341f8b-eb15-4664-bc71-7b10fa0be10c None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:29.115970 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-bd341f8b-eb15-4664-bc71-7b10fa0be10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:29.162972 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742aef4-3b99-453d-ab79-ca4961703fb4 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:29.170972 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c62a7a-f48c-47ce-95b6-5b8f5806b351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:29.170972 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c62a7a-f48c-47ce-95b6-5b8f5806b351 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:29.170972 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c62a7a-f48c-47ce-95b6-5b8f5806b351 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:29.170972 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c62a7a-f48c-47ce-95b6-5b8f5806b351 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:29.175581 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c62a7a-f48c-47ce-95b6-5b8f5806b351 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:29.175581 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c62a7a-f48c-47ce-95b6-5b8f5806b351 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:29.187064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1032/2070] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:29.192495 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd341f8b-eb15-4664-bc71-7b10fa0be10c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:29.196761 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1039/2071] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:29 2022] POST /identity/v3/users => generated 307 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:03:29.201301 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c62a7a-f48c-47ce-95b6-5b8f5806b351 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:29.207291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd341f8b-eb15-4664-bc71-7b10fa0be10c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:29.212746 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c62a7a-f48c-47ce-95b6-5b8f5806b351 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:29.212874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af6c35-da7a-4ec4-aaf5-65d2233ecf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:29.213523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af6c35-da7a-4ec4-aaf5-65d2233ecf13 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:29.213828 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af6c35-da7a-4ec4-aaf5-65d2233ecf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:29.214249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af6c35-da7a-4ec4-aaf5-65d2233ecf13 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:29.215014 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af6c35-da7a-4ec4-aaf5-65d2233ecf13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:03:29.215550 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af6c35-da7a-4ec4-aaf5-65d2233ecf13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:29.217168 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af6c35-da7a-4ec4-aaf5-65d2233ecf13 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:29.218246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b9447-95c4-410b-b84a-84a411c264cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:29.218922 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b9447-95c4-410b-b84a-84a411c264cc admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:29.219199 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b9447-95c4-410b-b84a-84a411c264cc admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:29.219473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b9447-95c4-410b-b84a-84a411c264cc admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:29.220195 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b9447-95c4-410b-b84a-84a411c264cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:29.221764 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b9447-95c4-410b-b84a-84a411c264cc admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:29.233746 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1033/2072] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:03:29.239679 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af6c35-da7a-4ec4-aaf5-65d2233ecf13 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:29.244397 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1040/2073] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:29.248853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b9447-95c4-410b-b84a-84a411c264cc admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:29.254890 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af6c35-da7a-4ec4-aaf5-65d2233ecf13 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:29.260501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83d168-135d-489f-9951-246c07e87e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:29.261156 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83d168-135d-489f-9951-246c07e87e6e admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:29.261614 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83d168-135d-489f-9951-246c07e87e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:29.261702 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83d168-135d-489f-9951-246c07e87e6e admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:29.262487 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa83d168-135d-489f-9951-246c07e87e6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:29.262971 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b9447-95c4-410b-b84a-84a411c264cc admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:29.265306 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa83d168-135d-489f-9951-246c07e87e6e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:29.269959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79e9c4-0d5a-4fa6-a7c5-77786ad7f952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:29.270936 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79e9c4-0d5a-4fa6-a7c5-77786ad7f952 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:29.271169 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79e9c4-0d5a-4fa6-a7c5-77786ad7f952 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:29.271430 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79e9c4-0d5a-4fa6-a7c5-77786ad7f952 admin admin] PATH_INFO: `/v3/projects/9ebe6358b6ab4e58a9fd05df062dbe98/users/d030e4506aeb40d9aaa221a625958531/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:29.297528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79e9c4-0d5a-4fa6-a7c5-77786ad7f952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ebe6358b6ab4e58a9fd05df062dbe98, user_id=d030e4506aeb40d9aaa221a625958531, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:29.301459 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1034/2074] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:29 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:03:29.302482 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79e9c4-0d5a-4fa6-a7c5-77786ad7f952 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:29.305029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83d168-135d-489f-9951-246c07e87e6e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:29.321082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1041/2075] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:29 2022] PUT /identity/v3/projects/9ebe6358b6ab4e58a9fd05df062dbe98/users/d030e4506aeb40d9aaa221a625958531/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:29.325951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79e9c4-0d5a-4fa6-a7c5-77786ad7f952 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:29.371232 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83d168-135d-489f-9951-246c07e87e6e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:29.394614 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79e9c4-0d5a-4fa6-a7c5-77786ad7f952 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:29.399795 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f6a07-52d7-4393-8d90-8b6e4667b418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:29.400462 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f6a07-52d7-4393-8d90-8b6e4667b418 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:29.400743 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f6a07-52d7-4393-8d90-8b6e4667b418 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:29.401015 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f6a07-52d7-4393-8d90-8b6e4667b418 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:29.401761 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f6a07-52d7-4393-8d90-8b6e4667b418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:29.403389 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f6a07-52d7-4393-8d90-8b6e4667b418 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:29.418073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1042/2076] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:29.422763 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f6a07-52d7-4393-8d90-8b6e4667b418 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:29.425256 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-547f6a07-52d7-4393-8d90-8b6e4667b418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:29.438148 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0637166-e64f-410a-886a-596a7f2502a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:29.438148 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0637166-e64f-410a-886a-596a7f2502a0 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:29.438148 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0637166-e64f-410a-886a-596a7f2502a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:29.438148 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0637166-e64f-410a-886a-596a7f2502a0 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:29.439482 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0637166-e64f-410a-886a-596a7f2502a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:29.442239 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0637166-e64f-410a-886a-596a7f2502a0 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:29.443415 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f6a07-52d7-4393-8d90-8b6e4667b418 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:29.450150 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a641f-61d4-444b-adcb-91c596cf2901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:29.451532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a641f-61d4-444b-adcb-91c596cf2901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:29.451766 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a641f-61d4-444b-adcb-91c596cf2901 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:29.452085 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a641f-61d4-444b-adcb-91c596cf2901 admin admin] PATH_INFO: `/v3/projects/91ec553ce72c4367b27985cfc1918e19` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:29.453451 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a641f-61d4-444b-adcb-91c596cf2901 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91ec553ce72c4367b27985cfc1918e19)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:29.456208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a641f-61d4-444b-adcb-91c596cf2901 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:29.471416 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1035/2077] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:29 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:03:29.472462 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0637166-e64f-410a-886a-596a7f2502a0 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:29.482960 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0637166-e64f-410a-886a-596a7f2502a0 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:29.489065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ab543-5817-4fcd-8261-ea1191d4e61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:29.490049 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ab543-5817-4fcd-8261-ea1191d4e61a admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:29.490388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ab543-5817-4fcd-8261-ea1191d4e61a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:29.490662 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ab543-5817-4fcd-8261-ea1191d4e61a admin admin] PATH_INFO: `/v3/projects/9ebe6358b6ab4e58a9fd05df062dbe98/users/d030e4506aeb40d9aaa221a625958531/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:29.492732 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ab543-5817-4fcd-8261-ea1191d4e61a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ebe6358b6ab4e58a9fd05df062dbe98, user_id=d030e4506aeb40d9aaa221a625958531, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:29.495563 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ab543-5817-4fcd-8261-ea1191d4e61a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:29.504662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c24a641f-61d4-444b-adcb-91c596cf2901 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91ec553ce72c4367b27985cfc1918e19', 'request_id': 'req-c24a641f-61d4-444b-adcb-91c596cf2901'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:29.505939 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1036/2078] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:29 2022] PUT /identity/v3/projects/9ebe6358b6ab4e58a9fd05df062dbe98/users/d030e4506aeb40d9aaa221a625958531/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:29.509900 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c24a641f-61d4-444b-adcb-91c596cf2901 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91ec553ce72c4367b27985cfc1918e19', 'request_id': 'req-c24a641f-61d4-444b-adcb-91c596cf2901'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:29.510828 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ab543-5817-4fcd-8261-ea1191d4e61a admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:29.516739 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c24a641f-61d4-444b-adcb-91c596cf2901 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91ec553ce72c4367b27985cfc1918e19', 'request_id': 'req-c24a641f-61d4-444b-adcb-91c596cf2901'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:29.529170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c24a641f-61d4-444b-adcb-91c596cf2901 admin admin] The token cache is being invalidate because project 91ec553ce72c4367b27985cfc1918e19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:29.529459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c24a641f-61d4-444b-adcb-91c596cf2901 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:29.530092 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c24a641f-61d4-444b-adcb-91c596cf2901 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:29.532224 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1043/2079] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:29 2022] DELETE /identity/v3/projects/91ec553ce72c4367b27985cfc1918e19 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:29.533508 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930305c-6476-4380-a50b-2ac0138d693d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:29.533795 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930305c-6476-4380-a50b-2ac0138d693d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:29.534070 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930305c-6476-4380-a50b-2ac0138d693d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:29.559566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-d930305c-6476-4380-a50b-2ac0138d693d None None] MFA Rules not processed for user `d030e4506aeb40d9aaa221a625958531`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:29.560753 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d930305c-6476-4380-a50b-2ac0138d693d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:29.580432 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ab543-5817-4fcd-8261-ea1191d4e61a admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:29.585285 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c90139-310a-48cc-866c-e56cbdbe98bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:29.585954 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c90139-310a-48cc-866c-e56cbdbe98bf admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:29.586394 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c90139-310a-48cc-866c-e56cbdbe98bf admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:29.586644 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c90139-310a-48cc-866c-e56cbdbe98bf admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:29.587368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c90139-310a-48cc-866c-e56cbdbe98bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:29.588953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c90139-310a-48cc-866c-e56cbdbe98bf admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:29.607674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1037/2080] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:29.607674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c90139-310a-48cc-866c-e56cbdbe98bf admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:29.609639 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-53c90139-310a-48cc-866c-e56cbdbe98bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:29.634052 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c90139-310a-48cc-866c-e56cbdbe98bf admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:29.646211 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeec9d7-acde-4261-9ce7-5b86c7482931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:29.647160 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeec9d7-acde-4261-9ce7-5b86c7482931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:29.647439 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeec9d7-acde-4261-9ce7-5b86c7482931 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:29.647761 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeec9d7-acde-4261-9ce7-5b86c7482931 admin admin] PATH_INFO: `/v3/users/6052214e89774d66a17e0425bfcb0965` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:29.648947 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeec9d7-acde-4261-9ce7-5b86c7482931 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6052214e89774d66a17e0425bfcb0965)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:29.651097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeec9d7-acde-4261-9ce7-5b86c7482931 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:29.663682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1044/2081] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:29 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:29.664788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930305c-6476-4380-a50b-2ac0138d693d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:29.668479 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d930305c-6476-4380-a50b-2ac0138d693d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:29.694230 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930305c-6476-4380-a50b-2ac0138d693d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:29.699288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158fe5d-7ad5-4c39-a0bd-5dd2ebf00631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:29.699951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158fe5d-7ad5-4c39-a0bd-5dd2ebf00631 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:29.700228 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158fe5d-7ad5-4c39-a0bd-5dd2ebf00631 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:29.700500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158fe5d-7ad5-4c39-a0bd-5dd2ebf00631 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:29.701250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e158fe5d-7ad5-4c39-a0bd-5dd2ebf00631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:29.702893 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e158fe5d-7ad5-4c39-a0bd-5dd2ebf00631 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:29.713018 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1045/2082] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:29.713727 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4eeec9d7-acde-4261-9ce7-5b86c7482931 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6052214e89774d66a17e0425bfcb0965'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:29.713978 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158fe5d-7ad5-4c39-a0bd-5dd2ebf00631 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:29.715815 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e158fe5d-7ad5-4c39-a0bd-5dd2ebf00631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:29.728029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4eeec9d7-acde-4261-9ce7-5b86c7482931 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6052214e89774d66a17e0425bfcb0965'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:29.729559 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4eeec9d7-acde-4261-9ce7-5b86c7482931 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6052214e89774d66a17e0425bfcb0965'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:29.749918 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4eeec9d7-acde-4261-9ce7-5b86c7482931 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6052214e89774d66a17e0425bfcb0965'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:29.751760 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4eeec9d7-acde-4261-9ce7-5b86c7482931 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6052214e89774d66a17e0425bfcb0965'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:29.778205 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1038/2083] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:29 2022] DELETE /identity/v3/users/6052214e89774d66a17e0425bfcb0965 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:29.783352 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeec9d7-acde-4261-9ce7-5b86c7482931 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:29.785087 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-4eeec9d7-acde-4261-9ce7-5b86c7482931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:29.809744 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158fe5d-7ad5-4c39-a0bd-5dd2ebf00631 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:29.879353 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeec9d7-acde-4261-9ce7-5b86c7482931 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:29.885350 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d205a231-a3c0-4f6a-8ce9-df11eaeafc12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:29.885515 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe9f6d-5a42-416d-91da-11f65c521413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:29.886233 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205a231-a3c0-4f6a-8ce9-df11eaeafc12 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:29.886406 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe9f6d-5a42-416d-91da-11f65c521413 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:29.886653 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205a231-a3c0-4f6a-8ce9-df11eaeafc12 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:29.886764 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe9f6d-5a42-416d-91da-11f65c521413 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:29.886949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe9f6d-5a42-416d-91da-11f65c521413 admin admin] PATH_INFO: `/v3/projects/536e26e8fce647c5a85862faf38a6854/users/61dc0b5438924b91a3bcf2440432589c/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:29.887131 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205a231-a3c0-4f6a-8ce9-df11eaeafc12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:29.889218 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d205a231-a3c0-4f6a-8ce9-df11eaeafc12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:29.904962 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe9f6d-5a42-416d-91da-11f65c521413 admin admin] RBAC: Authorizing `identity:create_grant(project_id=536e26e8fce647c5a85862faf38a6854, user_id=61dc0b5438924b91a3bcf2440432589c, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:29.908599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe9f6d-5a42-416d-91da-11f65c521413 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:29.926717 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1039/2084] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:29 2022] PUT /identity/v3/projects/536e26e8fce647c5a85862faf38a6854/users/61dc0b5438924b91a3bcf2440432589c/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:29.927772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe9f6d-5a42-416d-91da-11f65c521413 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:29.934206 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0dbe9f6d-5a42-416d-91da-11f65c521413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:29.965634 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d205a231-a3c0-4f6a-8ce9-df11eaeafc12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:29.967323 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d205a231-a3c0-4f6a-8ce9-df11eaeafc12 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:29.977776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1046/2085] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:29 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:29.979320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf221a-0f56-464e-9a91-b91651034a3d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:29.979552 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf221a-0f56-464e-9a91-b91651034a3d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:29.979799 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf221a-0f56-464e-9a91-b91651034a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:30.014353 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3bbf221a-0f56-464e-9a91-b91651034a3d None None] MFA Rules not processed for user `61dc0b5438924b91a3bcf2440432589c`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:30.016048 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3bbf221a-0f56-464e-9a91-b91651034a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:30.074109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe9f6d-5a42-416d-91da-11f65c521413 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:30.083882 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bde9f-b27a-4da0-931b-38ae16b37837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:30.084820 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bde9f-b27a-4da0-931b-38ae16b37837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:30.085119 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bde9f-b27a-4da0-931b-38ae16b37837 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:30.085399 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bde9f-b27a-4da0-931b-38ae16b37837 admin admin] PATH_INFO: `/v3/users/2f3aab2064584c95949ee6ce258cb16b` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:30.093401 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bde9f-b27a-4da0-931b-38ae16b37837 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f3aab2064584c95949ee6ce258cb16b)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:30.098754 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bde9f-b27a-4da0-931b-38ae16b37837 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:30.129328 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1047/2086] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:30.130539 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf221a-0f56-464e-9a91-b91651034a3d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:30.132908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3bbf221a-0f56-464e-9a91-b91651034a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:30.177864 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3d7bde9f-b27a-4da0-931b-38ae16b37837 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f3aab2064584c95949ee6ce258cb16b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:30.196068 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3d7bde9f-b27a-4da0-931b-38ae16b37837 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f3aab2064584c95949ee6ce258cb16b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:30.197236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3d7bde9f-b27a-4da0-931b-38ae16b37837 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f3aab2064584c95949ee6ce258cb16b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:30.213840 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3d7bde9f-b27a-4da0-931b-38ae16b37837 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f3aab2064584c95949ee6ce258cb16b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:30.214604 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3d7bde9f-b27a-4da0-931b-38ae16b37837 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f3aab2064584c95949ee6ce258cb16b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:30.231588 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf221a-0f56-464e-9a91-b91651034a3d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:30.243243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1040/2087] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:29 2022] DELETE /identity/v3/users/2f3aab2064584c95949ee6ce258cb16b => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:30.243243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bde9f-b27a-4da0-931b-38ae16b37837 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:30.243243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3d7bde9f-b27a-4da0-931b-38ae16b37837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:30.244427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89b10c-7fe3-4470-a669-49015fa30652 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:30.245092 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89b10c-7fe3-4470-a669-49015fa30652 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:30.245374 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89b10c-7fe3-4470-a669-49015fa30652 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:30.245648 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89b10c-7fe3-4470-a669-49015fa30652 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:30.247574 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6a89b10c-7fe3-4470-a669-49015fa30652 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:30.311306 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bde9f-b27a-4da0-931b-38ae16b37837 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:30.315821 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d49ae0-5535-47fa-9678-b855acc409aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:30.316662 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d49ae0-5535-47fa-9678-b855acc409aa service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:30.317029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d49ae0-5535-47fa-9678-b855acc409aa service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:30.317307 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d49ae0-5535-47fa-9678-b855acc409aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:30.319251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f1d49ae0-5535-47fa-9678-b855acc409aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:30.324866 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89b10c-7fe3-4470-a669-49015fa30652 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:30.327386 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89b10c-7fe3-4470-a669-49015fa30652 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:30.338183 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1048/2088] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:30.338841 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89b10c-7fe3-4470-a669-49015fa30652 service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:30.340162 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6a89b10c-7fe3-4470-a669-49015fa30652 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:30.357267 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89b10c-7fe3-4470-a669-49015fa30652 service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:30.401944 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d49ae0-5535-47fa-9678-b855acc409aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:30.404031 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d49ae0-5535-47fa-9678-b855acc409aa service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:30.413485 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1041/2089] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:30.414845 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042af07-9348-4aa6-be27-0ead7dd3b057 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:30.415142 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042af07-9348-4aa6-be27-0ead7dd3b057 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:30.415419 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042af07-9348-4aa6-be27-0ead7dd3b057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:30.429138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1be8e-c44f-4b9e-8087-0b83f7e2603d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:30.430076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1be8e-c44f-4b9e-8087-0b83f7e2603d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:30.430454 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1be8e-c44f-4b9e-8087-0b83f7e2603d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:30.430800 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1be8e-c44f-4b9e-8087-0b83f7e2603d admin admin] PATH_INFO: `/v3/projects/95ac9f70716d49aaaac1fb03684c32ac` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:30.432358 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d1be8e-c44f-4b9e-8087-0b83f7e2603d admin admin] RBAC: Authorizing `identity:delete_project(project_id=95ac9f70716d49aaaac1fb03684c32ac)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:30.434496 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-9042af07-9348-4aa6-be27-0ead7dd3b057 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:30.434629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d1be8e-c44f-4b9e-8087-0b83f7e2603d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:30.440137 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9042af07-9348-4aa6-be27-0ead7dd3b057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:30.448654 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1042/2090] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:30.449469 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042af07-9348-4aa6-be27-0ead7dd3b057 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:30.450928 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9042af07-9348-4aa6-be27-0ead7dd3b057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:30.465775 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042af07-9348-4aa6-be27-0ead7dd3b057 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:30.471635 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4f45d-272b-4153-b4c7-488197cf12fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:30.472607 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4f45d-272b-4153-b4c7-488197cf12fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:30.473008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4f45d-272b-4153-b4c7-488197cf12fc admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:30.473293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4f45d-272b-4153-b4c7-488197cf12fc admin admin] PATH_INFO: `/v3/users/61dc0b5438924b91a3bcf2440432589c` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:30.474554 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe4f45d-272b-4153-b4c7-488197cf12fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=61dc0b5438924b91a3bcf2440432589c)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:30.476589 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe4f45d-272b-4153-b4c7-488197cf12fc admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:30.495786 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-91d1be8e-c44f-4b9e-8087-0b83f7e2603d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95ac9f70716d49aaaac1fb03684c32ac', 'request_id': 'req-91d1be8e-c44f-4b9e-8087-0b83f7e2603d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:30.502216 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-91d1be8e-c44f-4b9e-8087-0b83f7e2603d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95ac9f70716d49aaaac1fb03684c32ac', 'request_id': 'req-91d1be8e-c44f-4b9e-8087-0b83f7e2603d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:30.510170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-91d1be8e-c44f-4b9e-8087-0b83f7e2603d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95ac9f70716d49aaaac1fb03684c32ac', 'request_id': 'req-91d1be8e-c44f-4b9e-8087-0b83f7e2603d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:30.524561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-91d1be8e-c44f-4b9e-8087-0b83f7e2603d admin admin] The token cache is being invalidate because project 95ac9f70716d49aaaac1fb03684c32ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:30.524986 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-91d1be8e-c44f-4b9e-8087-0b83f7e2603d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:30.525707 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-91d1be8e-c44f-4b9e-8087-0b83f7e2603d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:30.527477 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1049/2091] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:30 2022] DELETE /identity/v3/projects/95ac9f70716d49aaaac1fb03684c32ac => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:30.528340 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1be8e-c44f-4b9e-8087-0b83f7e2603d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:30.533823 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-91d1be8e-c44f-4b9e-8087-0b83f7e2603d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:30.549106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4fe4f45d-272b-4153-b4c7-488197cf12fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61dc0b5438924b91a3bcf2440432589c'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:30.557019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4fe4f45d-272b-4153-b4c7-488197cf12fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61dc0b5438924b91a3bcf2440432589c'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:30.557732 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4fe4f45d-272b-4153-b4c7-488197cf12fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61dc0b5438924b91a3bcf2440432589c'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:30.565783 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4fe4f45d-272b-4153-b4c7-488197cf12fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61dc0b5438924b91a3bcf2440432589c'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:30.566518 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4fe4f45d-272b-4153-b4c7-488197cf12fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61dc0b5438924b91a3bcf2440432589c'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:30.586748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1043/2092] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:30 2022] DELETE /identity/v3/users/61dc0b5438924b91a3bcf2440432589c => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:30.588331 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f57b7b-d0b9-46d5-b9c9-821cc77ea931 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:30.588619 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f57b7b-d0b9-46d5-b9c9-821cc77ea931 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:30.588894 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f57b7b-d0b9-46d5-b9c9-821cc77ea931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:30.600982 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1be8e-c44f-4b9e-8087-0b83f7e2603d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:30.608668 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-39f57b7b-d0b9-46d5-b9c9-821cc77ea931 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:30.610525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-39f57b7b-d0b9-46d5-b9c9-821cc77ea931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:30.679828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a730da44-12ea-40f1-bbb6-76a9e290deeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:30.680457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730da44-12ea-40f1-bbb6-76a9e290deeb service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:30.681434 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730da44-12ea-40f1-bbb6-76a9e290deeb service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:30.681725 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730da44-12ea-40f1-bbb6-76a9e290deeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:30.684204 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a730da44-12ea-40f1-bbb6-76a9e290deeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:30.690152 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1044/2093] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:30.691683 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfbdd7-5b90-44ba-91e8-c8b51d14e328 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:30.692056 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfbdd7-5b90-44ba-91e8-c8b51d14e328 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:30.692404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfbdd7-5b90-44ba-91e8-c8b51d14e328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:30.709278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a730da44-12ea-40f1-bbb6-76a9e290deeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:30.710792 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a730da44-12ea-40f1-bbb6-76a9e290deeb service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:30.713459 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-f0bfbdd7-5b90-44ba-91e8-c8b51d14e328 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:30.715409 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f0bfbdd7-5b90-44ba-91e8-c8b51d14e328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:30.726653 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1045/2094] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:30.728395 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c2146-b445-4d11-805e-14fef22ff01e None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:30.728688 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c2146-b445-4d11-805e-14fef22ff01e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:30.729034 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c2146-b445-4d11-805e-14fef22ff01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:30.736278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1050/2095] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:30.738231 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a730da44-12ea-40f1-bbb6-76a9e290deeb service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:30.739589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a730da44-12ea-40f1-bbb6-76a9e290deeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:30.750663 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-371c2146-b445-4d11-805e-14fef22ff01e None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:30.752286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-371c2146-b445-4d11-805e-14fef22ff01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:30.753732 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a730da44-12ea-40f1-bbb6-76a9e290deeb service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:30.760147 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e063732-c610-4cf3-86ab-54dfdb2bc3d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:30.761086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1046/2096] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:30.761220 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e063732-c610-4cf3-86ab-54dfdb2bc3d8 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:30.761304 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e063732-c610-4cf3-86ab-54dfdb2bc3d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:30.761613 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e063732-c610-4cf3-86ab-54dfdb2bc3d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:30.765267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c2146-b445-4d11-805e-14fef22ff01e None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:30.772515 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e063732-c610-4cf3-86ab-54dfdb2bc3d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:30.774439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e063732-c610-4cf3-86ab-54dfdb2bc3d8 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:30.780605 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c2146-b445-4d11-805e-14fef22ff01e None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:30.786437 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1051/2097] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 23:03:30.786709 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9038ce7-87f0-4259-b7d2-fd425a83e189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:30.787493 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9038ce7-87f0-4259-b7d2-fd425a83e189 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:30.787771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9038ce7-87f0-4259-b7d2-fd425a83e189 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:30.788117 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9038ce7-87f0-4259-b7d2-fd425a83e189 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:30.788967 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9038ce7-87f0-4259-b7d2-fd425a83e189 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:03:30.789294 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9038ce7-87f0-4259-b7d2-fd425a83e189 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:30.791042 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9038ce7-87f0-4259-b7d2-fd425a83e189 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:30.804006 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1047/2098] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:03:30.808334 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e063732-c610-4cf3-86ab-54dfdb2bc3d8 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:30.821077 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e063732-c610-4cf3-86ab-54dfdb2bc3d8 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:30.826147 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18326264-6ca3-438c-9f3e-932b5f11a534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:30.826875 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18326264-6ca3-438c-9f3e-932b5f11a534 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:30.827154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18326264-6ca3-438c-9f3e-932b5f11a534 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:30.827428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18326264-6ca3-438c-9f3e-932b5f11a534 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:30.828186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18326264-6ca3-438c-9f3e-932b5f11a534 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:30.830218 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18326264-6ca3-438c-9f3e-932b5f11a534 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:30.855293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1052/2099] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:30 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:30.860548 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9038ce7-87f0-4259-b7d2-fd425a83e189 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:30.870657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9038ce7-87f0-4259-b7d2-fd425a83e189 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:30.958829 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c37fd-eca7-4b98-b00f-6529269727f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:30.959522 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c37fd-eca7-4b98-b00f-6529269727f7 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:30.959804 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c37fd-eca7-4b98-b00f-6529269727f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:30.960106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c37fd-eca7-4b98-b00f-6529269727f7 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:30.960920 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c37fd-eca7-4b98-b00f-6529269727f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:30.963586 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c37fd-eca7-4b98-b00f-6529269727f7 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:30.986912 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1048/2100] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:30 2022] POST /identity/v3/users => generated 287 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:03:30.992139 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18326264-6ca3-438c-9f3e-932b5f11a534 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:31.002984 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18326264-6ca3-438c-9f3e-932b5f11a534 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:31.007950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88bede-0c7a-4f7a-b051-255b93f328d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:31.008579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88bede-0c7a-4f7a-b051-255b93f328d8 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:31.008983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88bede-0c7a-4f7a-b051-255b93f328d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:31.009310 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88bede-0c7a-4f7a-b051-255b93f328d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:31.010187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88bede-0c7a-4f7a-b051-255b93f328d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:31.011936 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88bede-0c7a-4f7a-b051-255b93f328d8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:31.033404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c37fd-eca7-4b98-b00f-6529269727f7 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:31.034925 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1053/2101] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:31.046095 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c37fd-eca7-4b98-b00f-6529269727f7 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:31.053352 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e448614-80b8-4ae1-9a1a-a1ef8d263fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:31.054641 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e448614-80b8-4ae1-9a1a-a1ef8d263fee admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:31.055254 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e448614-80b8-4ae1-9a1a-a1ef8d263fee admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:31.056207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e448614-80b8-4ae1-9a1a-a1ef8d263fee admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:31.056970 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e448614-80b8-4ae1-9a1a-a1ef8d263fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:31.058701 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e448614-80b8-4ae1-9a1a-a1ef8d263fee admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:31.073040 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1049/2102] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:31.076460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88bede-0c7a-4f7a-b051-255b93f328d8 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:31.088486 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88bede-0c7a-4f7a-b051-255b93f328d8 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:31.098334 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1632360-e969-41e3-ad78-3e1192565568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:31.103006 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1632360-e969-41e3-ad78-3e1192565568 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:31.103498 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1632360-e969-41e3-ad78-3e1192565568 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:31.103928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1632360-e969-41e3-ad78-3e1192565568 admin admin] PATH_INFO: `/v3/projects/cf0da59c36484adc972ff97534362090/users/c68f5f5c1f944f6a9b342134daa16c18/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:31.104302 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e448614-80b8-4ae1-9a1a-a1ef8d263fee admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:31.106949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0e448614-80b8-4ae1-9a1a-a1ef8d263fee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:31.125418 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e448614-80b8-4ae1-9a1a-a1ef8d263fee admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:31.132175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a36b2-b6ba-4b85-a0e7-fc0b283e9e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:31.132318 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1632360-e969-41e3-ad78-3e1192565568 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf0da59c36484adc972ff97534362090, user_id=c68f5f5c1f944f6a9b342134daa16c18, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:31.133307 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a36b2-b6ba-4b85-a0e7-fc0b283e9e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:31.133661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a36b2-b6ba-4b85-a0e7-fc0b283e9e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:31.134063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a36b2-b6ba-4b85-a0e7-fc0b283e9e41 admin admin] PATH_INFO: `/v3/projects/225886933ebb4238bfa5ed97514ff0f9` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:31.135492 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a36b2-b6ba-4b85-a0e7-fc0b283e9e41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=225886933ebb4238bfa5ed97514ff0f9)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:31.135969 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1632360-e969-41e3-ad78-3e1192565568 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:31.138228 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a36b2-b6ba-4b85-a0e7-fc0b283e9e41 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:31.152739 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1054/2103] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:31 2022] PUT /identity/v3/projects/cf0da59c36484adc972ff97534362090/users/c68f5f5c1f944f6a9b342134daa16c18/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:31.165183 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc3e30-5db6-4903-85bb-44caed10e032 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:31.165474 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc3e30-5db6-4903-85bb-44caed10e032 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:31.165875 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc3e30-5db6-4903-85bb-44caed10e032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:31.203371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-bcfc3e30-5db6-4903-85bb-44caed10e032 None None] MFA Rules not processed for user `c68f5f5c1f944f6a9b342134daa16c18`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:31.204627 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-bcfc3e30-5db6-4903-85bb-44caed10e032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:31.209438 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-679a36b2-b6ba-4b85-a0e7-fc0b283e9e41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '225886933ebb4238bfa5ed97514ff0f9', 'request_id': 'req-679a36b2-b6ba-4b85-a0e7-fc0b283e9e41'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:31.217531 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-679a36b2-b6ba-4b85-a0e7-fc0b283e9e41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '225886933ebb4238bfa5ed97514ff0f9', 'request_id': 'req-679a36b2-b6ba-4b85-a0e7-fc0b283e9e41'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:31.223720 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-679a36b2-b6ba-4b85-a0e7-fc0b283e9e41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '225886933ebb4238bfa5ed97514ff0f9', 'request_id': 'req-679a36b2-b6ba-4b85-a0e7-fc0b283e9e41'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:31.240851 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-679a36b2-b6ba-4b85-a0e7-fc0b283e9e41 admin admin] The token cache is being invalidate because project 225886933ebb4238bfa5ed97514ff0f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:31.241139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-679a36b2-b6ba-4b85-a0e7-fc0b283e9e41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:31.241788 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-679a36b2-b6ba-4b85-a0e7-fc0b283e9e41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:31.246285 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1050/2104] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:31 2022] DELETE /identity/v3/projects/225886933ebb4238bfa5ed97514ff0f9 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:31.246285 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a36b2-b6ba-4b85-a0e7-fc0b283e9e41 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:31.246285 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-679a36b2-b6ba-4b85-a0e7-fc0b283e9e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:31.289068 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1055/2105] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:03:31.290402 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07ede4-ad67-4119-a962-7fbc9bd03f38 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:31.290694 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07ede4-ad67-4119-a962-7fbc9bd03f38 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:31.290972 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07ede4-ad67-4119-a962-7fbc9bd03f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:31.309902 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-1f07ede4-ad67-4119-a962-7fbc9bd03f38 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:31.311767 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1f07ede4-ad67-4119-a962-7fbc9bd03f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:31.320638 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1056/2106] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:31.321277 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07ede4-ad67-4119-a962-7fbc9bd03f38 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:31.322573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1f07ede4-ad67-4119-a962-7fbc9bd03f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:31.326496 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a36b2-b6ba-4b85-a0e7-fc0b283e9e41 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:31.333338 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22c9fc-9423-4ee2-840a-ae78cc1e2ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:31.334712 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22c9fc-9423-4ee2-840a-ae78cc1e2ece admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:31.334991 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22c9fc-9423-4ee2-840a-ae78cc1e2ece admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:31.335395 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22c9fc-9423-4ee2-840a-ae78cc1e2ece admin admin] PATH_INFO: `/v3/projects/536e26e8fce647c5a85862faf38a6854` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:31.336588 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b22c9fc-9423-4ee2-840a-ae78cc1e2ece admin admin] RBAC: Authorizing `identity:delete_project(project_id=536e26e8fce647c5a85862faf38a6854)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:31.338763 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b22c9fc-9423-4ee2-840a-ae78cc1e2ece admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:31.380997 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07ede4-ad67-4119-a962-7fbc9bd03f38 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:31.385893 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8b22c9fc-9423-4ee2-840a-ae78cc1e2ece admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '536e26e8fce647c5a85862faf38a6854', 'request_id': 'req-8b22c9fc-9423-4ee2-840a-ae78cc1e2ece'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:31.392487 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8b22c9fc-9423-4ee2-840a-ae78cc1e2ece admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '536e26e8fce647c5a85862faf38a6854', 'request_id': 'req-8b22c9fc-9423-4ee2-840a-ae78cc1e2ece'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:31.397979 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8b22c9fc-9423-4ee2-840a-ae78cc1e2ece admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '536e26e8fce647c5a85862faf38a6854', 'request_id': 'req-8b22c9fc-9423-4ee2-840a-ae78cc1e2ece'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:31.412661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8b22c9fc-9423-4ee2-840a-ae78cc1e2ece admin admin] The token cache is being invalidate because project 536e26e8fce647c5a85862faf38a6854 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:31.412882 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8b22c9fc-9423-4ee2-840a-ae78cc1e2ece admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:31.413536 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8b22c9fc-9423-4ee2-840a-ae78cc1e2ece admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:31.415690 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1051/2107] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:31 2022] DELETE /identity/v3/projects/536e26e8fce647c5a85862faf38a6854 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:31.422023 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c318f-255d-45f2-828c-f59537797f9c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:31.422023 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c318f-255d-45f2-828c-f59537797f9c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:31.422023 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c318f-255d-45f2-828c-f59537797f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:31.445530 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-8e9c318f-255d-45f2-828c-f59537797f9c None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:31.446332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba6035-3e82-46df-9e04-1cfd39db5b56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:31.446756 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8e9c318f-255d-45f2-828c-f59537797f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:31.447163 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba6035-3e82-46df-9e04-1cfd39db5b56 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:31.447578 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba6035-3e82-46df-9e04-1cfd39db5b56 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:31.447859 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba6035-3e82-46df-9e04-1cfd39db5b56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:31.449615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-59ba6035-3e82-46df-9e04-1cfd39db5b56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:31.514691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba6035-3e82-46df-9e04-1cfd39db5b56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:31.516674 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba6035-3e82-46df-9e04-1cfd39db5b56 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:31.521772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1052/2108] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:31.523220 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439813d-4f4b-47d3-a3bc-ec5d550f8bde None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:31.523501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439813d-4f4b-47d3-a3bc-ec5d550f8bde None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:31.523766 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439813d-4f4b-47d3-a3bc-ec5d550f8bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:31.527768 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1057/2109] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:31.528874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba6035-3e82-46df-9e04-1cfd39db5b56 service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:31.543892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d439813d-4f4b-47d3-a3bc-ec5d550f8bde None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:31.544295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba6035-3e82-46df-9e04-1cfd39db5b56 service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:31.545079 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d439813d-4f4b-47d3-a3bc-ec5d550f8bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:31.550094 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9110f4c-eabd-4c84-bbdd-a8505fb19dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:31.550788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9110f4c-eabd-4c84-bbdd-a8505fb19dab admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:31.551075 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9110f4c-eabd-4c84-bbdd-a8505fb19dab admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:31.551354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9110f4c-eabd-4c84-bbdd-a8505fb19dab admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:31.552109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9110f4c-eabd-4c84-bbdd-a8505fb19dab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:03:31.552386 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9110f4c-eabd-4c84-bbdd-a8505fb19dab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:31.553980 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9110f4c-eabd-4c84-bbdd-a8505fb19dab admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:31.557063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1053/2110] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:31.557903 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439813d-4f4b-47d3-a3bc-ec5d550f8bde None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:31.559524 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d439813d-4f4b-47d3-a3bc-ec5d550f8bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:31.567460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1058/2111] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:03:31.568857 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee017bca-1547-4df8-b07c-ec07f62bb327 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:31.569150 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee017bca-1547-4df8-b07c-ec07f62bb327 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:31.569433 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee017bca-1547-4df8-b07c-ec07f62bb327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:31.582353 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439813d-4f4b-47d3-a3bc-ec5d550f8bde None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:31.589693 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-ee017bca-1547-4df8-b07c-ec07f62bb327 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:31.591051 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ee017bca-1547-4df8-b07c-ec07f62bb327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:31.596592 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aff724-5cb1-4d49-995d-73f1ad6af26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:31.597499 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aff724-5cb1-4d49-995d-73f1ad6af26c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:31.597782 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aff724-5cb1-4d49-995d-73f1ad6af26c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:31.598055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aff724-5cb1-4d49-995d-73f1ad6af26c admin admin] PATH_INFO: `/v3/users/c68f5f5c1f944f6a9b342134daa16c18` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:31.599285 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aff724-5cb1-4d49-995d-73f1ad6af26c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c68f5f5c1f944f6a9b342134daa16c18)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:31.600153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1059/2112] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:31.601012 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee017bca-1547-4df8-b07c-ec07f62bb327 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:31.601379 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aff724-5cb1-4d49-995d-73f1ad6af26c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:31.611228 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee017bca-1547-4df8-b07c-ec07f62bb327 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:31.616191 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef68cd7-ad47-4af0-b404-efa42453a038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:31.616833 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef68cd7-ad47-4af0-b404-efa42453a038 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:31.617114 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef68cd7-ad47-4af0-b404-efa42453a038 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:31.617392 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef68cd7-ad47-4af0-b404-efa42453a038 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:31.618235 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef68cd7-ad47-4af0-b404-efa42453a038 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:31.620128 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef68cd7-ad47-4af0-b404-efa42453a038 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:31.652215 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1060/2113] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:31 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:31.653047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef68cd7-ad47-4af0-b404-efa42453a038 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:31.664865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-72aff724-5cb1-4d49-995d-73f1ad6af26c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c68f5f5c1f944f6a9b342134daa16c18'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:31.672458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-72aff724-5cb1-4d49-995d-73f1ad6af26c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c68f5f5c1f944f6a9b342134daa16c18'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:31.673142 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-72aff724-5cb1-4d49-995d-73f1ad6af26c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c68f5f5c1f944f6a9b342134daa16c18'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:31.680616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-72aff724-5cb1-4d49-995d-73f1ad6af26c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c68f5f5c1f944f6a9b342134daa16c18'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:31.681246 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-72aff724-5cb1-4d49-995d-73f1ad6af26c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c68f5f5c1f944f6a9b342134daa16c18'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:31.698450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1054/2114] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:31 2022] DELETE /identity/v3/users/c68f5f5c1f944f6a9b342134daa16c18 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:31.703403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aff724-5cb1-4d49-995d-73f1ad6af26c admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:31.706334 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-72aff724-5cb1-4d49-995d-73f1ad6af26c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:31.738719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef68cd7-ad47-4af0-b404-efa42453a038 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:31.784363 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aff724-5cb1-4d49-995d-73f1ad6af26c admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:31.789586 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963f187-af31-4c3d-9871-eadf687b9895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:31.790166 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963f187-af31-4c3d-9871-eadf687b9895 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:31.790446 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963f187-af31-4c3d-9871-eadf687b9895 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:31.790716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963f187-af31-4c3d-9871-eadf687b9895 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:31.791449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963f187-af31-4c3d-9871-eadf687b9895 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:31.793362 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963f187-af31-4c3d-9871-eadf687b9895 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:31.804537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b517641-ad2a-4d6b-a012-5dbd99c2d850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:31.805402 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b517641-ad2a-4d6b-a012-5dbd99c2d850 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:31.805765 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b517641-ad2a-4d6b-a012-5dbd99c2d850 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:31.806251 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b517641-ad2a-4d6b-a012-5dbd99c2d850 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:31.806931 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b517641-ad2a-4d6b-a012-5dbd99c2d850 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:03:31.818031 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b517641-ad2a-4d6b-a012-5dbd99c2d850 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:31.818031 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b517641-ad2a-4d6b-a012-5dbd99c2d850 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:31.818406 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1055/2115] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:31 2022] POST /identity/v3/users => generated 286 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:31.818406 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c8b0d-71a4-49df-9eea-32f9e0a21f3b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:31.818406 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c8b0d-71a4-49df-9eea-32f9e0a21f3b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:31.818406 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c8b0d-71a4-49df-9eea-32f9e0a21f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:31.826269 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1061/2116] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:03:31.826269 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b517641-ad2a-4d6b-a012-5dbd99c2d850 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:31.835825 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b517641-ad2a-4d6b-a012-5dbd99c2d850 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:31.839889 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-bf4c8b0d-71a4-49df-9eea-32f9e0a21f3b None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:31.840709 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222bd79b-343d-4c99-bd23-07490e90204e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:31.841360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222bd79b-343d-4c99-bd23-07490e90204e admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:31.841639 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222bd79b-343d-4c99-bd23-07490e90204e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:31.841888 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-bf4c8b0d-71a4-49df-9eea-32f9e0a21f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:31.842001 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222bd79b-343d-4c99-bd23-07490e90204e admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:31.842653 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222bd79b-343d-4c99-bd23-07490e90204e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:31.844768 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222bd79b-343d-4c99-bd23-07490e90204e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:31.852184 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1056/2117] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:31.853163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c8b0d-71a4-49df-9eea-32f9e0a21f3b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:31.855171 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-bf4c8b0d-71a4-49df-9eea-32f9e0a21f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:31.864290 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1062/2118] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:31.866720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222bd79b-343d-4c99-bd23-07490e90204e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:31.868544 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-222bd79b-343d-4c99-bd23-07490e90204e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:31.870575 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c8b0d-71a4-49df-9eea-32f9e0a21f3b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:31.876031 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d679c-f8e8-40ab-9cb7-51dea1ddd5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:31.876765 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d679c-f8e8-40ab-9cb7-51dea1ddd5cf admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:31.877110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d679c-f8e8-40ab-9cb7-51dea1ddd5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:31.877449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d679c-f8e8-40ab-9cb7-51dea1ddd5cf admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:31.878243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d679c-f8e8-40ab-9cb7-51dea1ddd5cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:31.880250 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d679c-f8e8-40ab-9cb7-51dea1ddd5cf admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:31.918204 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1057/2119] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:31 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:31.919292 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d679c-f8e8-40ab-9cb7-51dea1ddd5cf admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:31.946401 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222bd79b-343d-4c99-bd23-07490e90204e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:32.008009 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d679c-f8e8-40ab-9cb7-51dea1ddd5cf admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:32.012791 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a37b3-8669-4e92-9895-54b205c4fcb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:32.013656 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a37b3-8669-4e92-9895-54b205c4fcb3 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:32.014298 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a37b3-8669-4e92-9895-54b205c4fcb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:32.014611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a37b3-8669-4e92-9895-54b205c4fcb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:32.015959 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a37b3-8669-4e92-9895-54b205c4fcb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:32.018022 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a37b3-8669-4e92-9895-54b205c4fcb3 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:32.020512 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42800381-5500-4f3a-81d0-0f80a4061976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:32.021262 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42800381-5500-4f3a-81d0-0f80a4061976 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:32.021566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42800381-5500-4f3a-81d0-0f80a4061976 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:32.021932 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42800381-5500-4f3a-81d0-0f80a4061976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:32.028666 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1058/2120] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:32.032594 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a37b3-8669-4e92-9895-54b205c4fcb3 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:32.032873 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42800381-5500-4f3a-81d0-0f80a4061976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:32.034734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42800381-5500-4f3a-81d0-0f80a4061976 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:32.044376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a37b3-8669-4e92-9895-54b205c4fcb3 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:32.049807 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1063/2121] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:32.051090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42800381-5500-4f3a-81d0-0f80a4061976 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:32.061174 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcda5a3-a897-4a61-922e-fb3c4b1801e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:32.061865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcda5a3-a897-4a61-922e-fb3c4b1801e1 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:32.062725 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcda5a3-a897-4a61-922e-fb3c4b1801e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:32.063349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcda5a3-a897-4a61-922e-fb3c4b1801e1 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:32.064052 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcda5a3-a897-4a61-922e-fb3c4b1801e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:32.067360 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcda5a3-a897-4a61-922e-fb3c4b1801e1 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:32.079075 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42800381-5500-4f3a-81d0-0f80a4061976 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:32.085182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37c501-7bad-4efb-9c0b-08738b1a7b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:32.086693 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37c501-7bad-4efb-9c0b-08738b1a7b54 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:32.087214 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37c501-7bad-4efb-9c0b-08738b1a7b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:32.087687 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37c501-7bad-4efb-9c0b-08738b1a7b54 admin admin] PATH_INFO: `/v3/projects/d55914be74874d4fb313c88e0c804e08/users/c9a9418ba4d643e5922979677d757cf7/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:32.101498 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1059/2122] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:32 2022] POST /identity/v3/users => generated 292 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:32.106914 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcda5a3-a897-4a61-922e-fb3c4b1801e1 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:32.117214 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37c501-7bad-4efb-9c0b-08738b1a7b54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d55914be74874d4fb313c88e0c804e08, user_id=c9a9418ba4d643e5922979677d757cf7, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:32.121467 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37c501-7bad-4efb-9c0b-08738b1a7b54 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:32.129880 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcda5a3-a897-4a61-922e-fb3c4b1801e1 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:32.135728 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00412717-b204-4bc9-944f-2aaf0209bec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:32.136918 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00412717-b204-4bc9-944f-2aaf0209bec7 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:32.137493 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00412717-b204-4bc9-944f-2aaf0209bec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:32.138010 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00412717-b204-4bc9-944f-2aaf0209bec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:32.139089 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00412717-b204-4bc9-944f-2aaf0209bec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:32.141310 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00412717-b204-4bc9-944f-2aaf0209bec7 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:32.146372 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1064/2123] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:32 2022] PUT /identity/v3/projects/d55914be74874d4fb313c88e0c804e08/users/c9a9418ba4d643e5922979677d757cf7/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:32.156075 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1060/2124] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:32.157098 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fd164-3fef-4ab0-95dd-065d3b77a797 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:32.157393 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fd164-3fef-4ab0-95dd-065d3b77a797 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:32.158608 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fd164-3fef-4ab0-95dd-065d3b77a797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:32.161361 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00412717-b204-4bc9-944f-2aaf0209bec7 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:32.174620 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00412717-b204-4bc9-944f-2aaf0209bec7 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:32.197240 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f53fd164-3fef-4ab0-95dd-065d3b77a797 None None] MFA Rules not processed for user `c9a9418ba4d643e5922979677d757cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:32.199892 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f53fd164-3fef-4ab0-95dd-065d3b77a797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:32.291721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1065/2125] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:32.292197 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63dc96f-e2d7-4356-9e72-60d2d3653880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:32.292197 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dc96f-e2d7-4356-9e72-60d2d3653880 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:32.292525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dc96f-e2d7-4356-9e72-60d2d3653880 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:32.293282 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dc96f-e2d7-4356-9e72-60d2d3653880 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:32.294195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63dc96f-e2d7-4356-9e72-60d2d3653880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:32.296960 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63dc96f-e2d7-4356-9e72-60d2d3653880 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:32.300642 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53fd164-3fef-4ab0-95dd-065d3b77a797 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:32.302072 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f53fd164-3fef-4ab0-95dd-065d3b77a797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:32.317405 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1061/2126] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:32.318977 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63dc96f-e2d7-4356-9e72-60d2d3653880 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:32.332427 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63dc96f-e2d7-4356-9e72-60d2d3653880 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:32.351682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55876a2b-f693-4b65-9efb-249376ca6c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:32.351682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55876a2b-f693-4b65-9efb-249376ca6c25 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:32.351682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55876a2b-f693-4b65-9efb-249376ca6c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:32.351682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55876a2b-f693-4b65-9efb-249376ca6c25 admin admin] PATH_INFO: `/v3/projects/c0c9afa417d540428c522e6658b7f90e/users/502516b956344e4191bab16c4338698a/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:32.382432 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55876a2b-f693-4b65-9efb-249376ca6c25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0c9afa417d540428c522e6658b7f90e, user_id=502516b956344e4191bab16c4338698a, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:32.387528 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55876a2b-f693-4b65-9efb-249376ca6c25 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:32.399901 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53fd164-3fef-4ab0-95dd-065d3b77a797 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:32.406021 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c04602-9318-4ae9-97c3-9bcb0de783ca service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81caf0558f2449cbbc8f4cd9446f363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:32.407894 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c04602-9318-4ae9-97c3-9bcb0de783ca service glance] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:32.408130 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c04602-9318-4ae9-97c3-9bcb0de783ca service glance] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:32.408408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c04602-9318-4ae9-97c3-9bcb0de783ca service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:32.418651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1062/2127] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:32 2022] PUT /identity/v3/projects/c0c9afa417d540428c522e6658b7f90e/users/502516b956344e4191bab16c4338698a/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:32.419938 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55876a2b-f693-4b65-9efb-249376ca6c25 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:32.430209 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-55876a2b-f693-4b65-9efb-249376ca6c25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:32.434364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c04602-9318-4ae9-97c3-9bcb0de783ca service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:32.435969 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c04602-9318-4ae9-97c3-9bcb0de783ca service glance] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:32.448481 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1066/2128] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:32.449860 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9508d-11ef-4650-a06a-e0d6272bdf71 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:32.450246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9508d-11ef-4650-a06a-e0d6272bdf71 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:32.450588 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9508d-11ef-4650-a06a-e0d6272bdf71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:32.500029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-99f9508d-11ef-4650-a06a-e0d6272bdf71 None None] MFA Rules not processed for user `502516b956344e4191bab16c4338698a`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:32.501245 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-99f9508d-11ef-4650-a06a-e0d6272bdf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:32.565238 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55876a2b-f693-4b65-9efb-249376ca6c25 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:32.579902 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee31ac-de23-4d63-b5d1-4313ed124b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:32.581339 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee31ac-de23-4d63-b5d1-4313ed124b42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:32.581770 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee31ac-de23-4d63-b5d1-4313ed124b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:32.582040 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee31ac-de23-4d63-b5d1-4313ed124b42 admin admin] PATH_INFO: `/v3/projects/cf0da59c36484adc972ff97534362090` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:32.587171 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee31ac-de23-4d63-b5d1-4313ed124b42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf0da59c36484adc972ff97534362090)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:32.589499 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee31ac-de23-4d63-b5d1-4313ed124b42 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:32.604273 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1067/2129] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:32.618005 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9508d-11ef-4650-a06a-e0d6272bdf71 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:32.627195 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-99f9508d-11ef-4650-a06a-e0d6272bdf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:32.668726 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-39ee31ac-de23-4d63-b5d1-4313ed124b42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf0da59c36484adc972ff97534362090', 'request_id': 'req-39ee31ac-de23-4d63-b5d1-4313ed124b42'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:32.679809 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-39ee31ac-de23-4d63-b5d1-4313ed124b42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf0da59c36484adc972ff97534362090', 'request_id': 'req-39ee31ac-de23-4d63-b5d1-4313ed124b42'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:32.689655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-39ee31ac-de23-4d63-b5d1-4313ed124b42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf0da59c36484adc972ff97534362090', 'request_id': 'req-39ee31ac-de23-4d63-b5d1-4313ed124b42'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:32.706721 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-39ee31ac-de23-4d63-b5d1-4313ed124b42 admin admin] The token cache is being invalidate because project cf0da59c36484adc972ff97534362090 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:32.707019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-39ee31ac-de23-4d63-b5d1-4313ed124b42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:32.707688 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-39ee31ac-de23-4d63-b5d1-4313ed124b42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:32.709847 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1063/2130] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:32 2022] DELETE /identity/v3/projects/cf0da59c36484adc972ff97534362090 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:32.714531 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc308c-6e51-481d-ae8f-ff3fc470f014 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:32.714816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc308c-6e51-481d-ae8f-ff3fc470f014 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:32.715089 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc308c-6e51-481d-ae8f-ff3fc470f014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:32.732455 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9508d-11ef-4650-a06a-e0d6272bdf71 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:32.735340 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-77dc308c-6e51-481d-ae8f-ff3fc470f014 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:32.737980 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-77dc308c-6e51-481d-ae8f-ff3fc470f014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:32.800225 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bcb01a-272c-4e97-83ad-af6301202ffd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:32.800952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bcb01a-272c-4e97-83ad-af6301202ffd service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:32.801336 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bcb01a-272c-4e97-83ad-af6301202ffd service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:32.801708 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bcb01a-272c-4e97-83ad-af6301202ffd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:32.803843 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-10bcb01a-272c-4e97-83ad-af6301202ffd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:32.846544 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1064/2131] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:03:32.854412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abaa97b-cfb4-48fc-96e9-c48fde727920 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:32.854920 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abaa97b-cfb4-48fc-96e9-c48fde727920 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:32.855370 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abaa97b-cfb4-48fc-96e9-c48fde727920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:32.893131 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-3abaa97b-cfb4-48fc-96e9-c48fde727920 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:32.894677 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3abaa97b-cfb4-48fc-96e9-c48fde727920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:32.900524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bcb01a-272c-4e97-83ad-af6301202ffd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:32.901969 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bcb01a-272c-4e97-83ad-af6301202ffd service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:32.907373 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1065/2132] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:03:32.910138 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abaa97b-cfb4-48fc-96e9-c48fde727920 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:32.916751 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1068/2133] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 23:03:32.924558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abaa97b-cfb4-48fc-96e9-c48fde727920 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:32.930026 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1d375-22c7-4ed6-98bd-0fe74a665321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:32.930848 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1d375-22c7-4ed6-98bd-0fe74a665321 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:32.931133 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1d375-22c7-4ed6-98bd-0fe74a665321 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:32.931405 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1d375-22c7-4ed6-98bd-0fe74a665321 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:32.932256 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed1d375-22c7-4ed6-98bd-0fe74a665321 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:03:32.932572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed1d375-22c7-4ed6-98bd-0fe74a665321 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:32.934630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed1d375-22c7-4ed6-98bd-0fe74a665321 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:32.950462 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1066/2134] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:03:32.954657 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bcb01a-272c-4e97-83ad-af6301202ffd service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:32.967154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bcb01a-272c-4e97-83ad-af6301202ffd service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:32.976863 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de528b-cd95-442f-8865-a6e36f603ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:32.979223 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de528b-cd95-442f-8865-a6e36f603ee2 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:32.979492 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de528b-cd95-442f-8865-a6e36f603ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:32.981266 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de528b-cd95-442f-8865-a6e36f603ee2 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:32.982368 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de528b-cd95-442f-8865-a6e36f603ee2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:32.987042 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de528b-cd95-442f-8865-a6e36f603ee2 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:33.022553 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1069/2135] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:32 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:33.026907 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1d375-22c7-4ed6-98bd-0fe74a665321 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:33.040768 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1d375-22c7-4ed6-98bd-0fe74a665321 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:33.139282 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d20a24-7acc-4cf2-a8eb-9937128ceacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:33.140183 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d20a24-7acc-4cf2-a8eb-9937128ceacf admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:33.140871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d20a24-7acc-4cf2-a8eb-9937128ceacf admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:33.141288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d20a24-7acc-4cf2-a8eb-9937128ceacf admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:33.142210 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d20a24-7acc-4cf2-a8eb-9937128ceacf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:33.144576 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d20a24-7acc-4cf2-a8eb-9937128ceacf admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:33.181451 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de528b-cd95-442f-8865-a6e36f603ee2 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:33.186773 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1067/2136] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:33 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:03:33.196033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de528b-cd95-442f-8865-a6e36f603ee2 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:33.199963 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68898d8c-bcf7-47b2-bb9f-4baee5a35904 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:33.200884 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68898d8c-bcf7-47b2-bb9f-4baee5a35904 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:33.206078 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68898d8c-bcf7-47b2-bb9f-4baee5a35904 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:33.208102 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1068/2137] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 9 23:03:33 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 23:03:33.215068 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de51dee-1279-4174-a9f4-5dccd62dc626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:33.215867 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de51dee-1279-4174-a9f4-5dccd62dc626 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:33.217691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de51dee-1279-4174-a9f4-5dccd62dc626 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:33.218052 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de51dee-1279-4174-a9f4-5dccd62dc626 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:33.219451 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de51dee-1279-4174-a9f4-5dccd62dc626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:33.222527 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc77fd-1274-4be6-b75d-f6fe166f6b14 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:33.222900 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc77fd-1274-4be6-b75d-f6fe166f6b14 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:33.223178 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc77fd-1274-4be6-b75d-f6fe166f6b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:33.238993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de51dee-1279-4174-a9f4-5dccd62dc626 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:33.261461 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b2bc77fd-1274-4be6-b75d-f6fe166f6b14 None None] MFA Rules not processed for user `02fcd7cc86134ce488c7dd75dfb174b8`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:33.262694 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b2bc77fd-1274-4be6-b75d-f6fe166f6b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:33.270159 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1070/2138] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:33.273012 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de51dee-1279-4174-a9f4-5dccd62dc626 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:33.302951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de51dee-1279-4174-a9f4-5dccd62dc626 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:33.308100 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b829c-8776-4392-b4f3-42ac94768fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:33.308710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b829c-8776-4392-b4f3-42ac94768fdc admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:33.308991 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b829c-8776-4392-b4f3-42ac94768fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:33.309269 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b829c-8776-4392-b4f3-42ac94768fdc admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:33.309989 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b829c-8776-4392-b4f3-42ac94768fdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:33.311634 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b829c-8776-4392-b4f3-42ac94768fdc admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:33.325602 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1071/2139] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:33.329712 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b829c-8776-4392-b4f3-42ac94768fdc admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:33.356071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b829c-8776-4392-b4f3-42ac94768fdc admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:33.363810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa0cbc-8eff-401f-9be2-27b7e33233e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:33.364824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa0cbc-8eff-401f-9be2-27b7e33233e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:33.365109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa0cbc-8eff-401f-9be2-27b7e33233e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:33.365524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa0cbc-8eff-401f-9be2-27b7e33233e2 admin admin] PATH_INFO: `/v3/projects/458f8b87a0b84582a8ab8f17d39fad8d/users/1be6a58da903448c973fd2862707cbb7/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:33.370377 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1069/2140] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 9 23:03:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:33.378308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc77fd-1274-4be6-b75d-f6fe166f6b14 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:33.380915 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b2bc77fd-1274-4be6-b75d-f6fe166f6b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:33.399004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc77fd-1274-4be6-b75d-f6fe166f6b14 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:33.407189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa0cbc-8eff-401f-9be2-27b7e33233e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=458f8b87a0b84582a8ab8f17d39fad8d, user_id=1be6a58da903448c973fd2862707cbb7, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:33.410461 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa0cbc-8eff-401f-9be2-27b7e33233e2 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:33.417456 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7168ddd8-75c4-422e-8b91-09e9e598a4de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:33.418221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168ddd8-75c4-422e-8b91-09e9e598a4de service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:33.418484 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168ddd8-75c4-422e-8b91-09e9e598a4de service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:33.418812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168ddd8-75c4-422e-8b91-09e9e598a4de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:33.430927 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1072/2141] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:33 2022] PUT /identity/v3/projects/458f8b87a0b84582a8ab8f17d39fad8d/users/1be6a58da903448c973fd2862707cbb7/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:03:33.434151 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7168ddd8-75c4-422e-8b91-09e9e598a4de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:33.435756 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7168ddd8-75c4-422e-8b91-09e9e598a4de service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:33.442150 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a2b37-f664-4ed9-a966-2547851b6b72 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:33.442445 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a2b37-f664-4ed9-a966-2547851b6b72 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:33.442723 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a2b37-f664-4ed9-a966-2547851b6b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:33.463981 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1070/2142] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:33.482726 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-939a2b37-f664-4ed9-a966-2547851b6b72 None None] MFA Rules not processed for user `1be6a58da903448c973fd2862707cbb7`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:33.484335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-939a2b37-f664-4ed9-a966-2547851b6b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:33.561058 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1073/2143] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:03:33.577101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7168ddd8-75c4-422e-8b91-09e9e598a4de service nova] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:33.579234 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7168ddd8-75c4-422e-8b91-09e9e598a4de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:33.638807 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7168ddd8-75c4-422e-8b91-09e9e598a4de service nova] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:33.710000 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa03471-683b-4743-8288-64365304827b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:33.710874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa03471-683b-4743-8288-64365304827b service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:33.711238 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa03471-683b-4743-8288-64365304827b service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:33.712169 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa03471-683b-4743-8288-64365304827b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:33.721605 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa03471-683b-4743-8288-64365304827b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:33.723211 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa03471-683b-4743-8288-64365304827b service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:33.732847 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1071/2144] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:33.759470 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a2b37-f664-4ed9-a966-2547851b6b72 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:33.831725 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a2b37-f664-4ed9-a966-2547851b6b72 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:33.838340 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc3e87-763d-4d5b-a8cd-d6c7d76e932f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:33.839254 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc3e87-763d-4d5b-a8cd-d6c7d76e932f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:33.839535 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc3e87-763d-4d5b-a8cd-d6c7d76e932f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:33.839809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc3e87-763d-4d5b-a8cd-d6c7d76e932f admin admin] PATH_INFO: `/v3/users/1be6a58da903448c973fd2862707cbb7` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:33.841006 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc3e87-763d-4d5b-a8cd-d6c7d76e932f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1be6a58da903448c973fd2862707cbb7)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:33.843366 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc3e87-763d-4d5b-a8cd-d6c7d76e932f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:33.890491 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-54dc3e87-763d-4d5b-a8cd-d6c7d76e932f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1be6a58da903448c973fd2862707cbb7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:33.897490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-54dc3e87-763d-4d5b-a8cd-d6c7d76e932f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1be6a58da903448c973fd2862707cbb7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:33.907409 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-54dc3e87-763d-4d5b-a8cd-d6c7d76e932f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1be6a58da903448c973fd2862707cbb7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:33.908193 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-54dc3e87-763d-4d5b-a8cd-d6c7d76e932f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1be6a58da903448c973fd2862707cbb7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:33.920499 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-54dc3e87-763d-4d5b-a8cd-d6c7d76e932f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1be6a58da903448c973fd2862707cbb7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:33.927045 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1074/2145] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:33 2022] DELETE /identity/v3/users/1be6a58da903448c973fd2862707cbb7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:33.931584 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb8b82-f85b-4e12-a2a5-e925c0e9fe82 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:33.931936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb8b82-f85b-4e12-a2a5-e925c0e9fe82 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:33.932304 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb8b82-f85b-4e12-a2a5-e925c0e9fe82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:33.950047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b8eb8b82-f85b-4e12-a2a5-e925c0e9fe82 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:33.951225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b8eb8b82-f85b-4e12-a2a5-e925c0e9fe82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:34.012256 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1072/2146] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:03:34.023108 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc3e87-763d-4d5b-a8cd-d6c7d76e932f admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:34.024965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-54dc3e87-763d-4d5b-a8cd-d6c7d76e932f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:34.093460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc3e87-763d-4d5b-a8cd-d6c7d76e932f admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:34.098503 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0ae0f-2852-4280-90b7-8ca6b81d30a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:34.099165 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0ae0f-2852-4280-90b7-8ca6b81d30a4 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:34.099447 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0ae0f-2852-4280-90b7-8ca6b81d30a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:34.099726 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0ae0f-2852-4280-90b7-8ca6b81d30a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:34.110595 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0ae0f-2852-4280-90b7-8ca6b81d30a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:34.112027 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0ae0f-2852-4280-90b7-8ca6b81d30a4 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:34.121882 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1075/2147] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:34.485604 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb8b82-f85b-4e12-a2a5-e925c0e9fe82 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:34.487417 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b8eb8b82-f85b-4e12-a2a5-e925c0e9fe82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:34.501921 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb8b82-f85b-4e12-a2a5-e925c0e9fe82 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:34.506556 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d4aa28-b4b7-46cc-8cde-9db1e7edd7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:34.507517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4aa28-b4b7-46cc-8cde-9db1e7edd7c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:34.507804 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4aa28-b4b7-46cc-8cde-9db1e7edd7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:34.508080 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4aa28-b4b7-46cc-8cde-9db1e7edd7c0 admin admin] PATH_INFO: `/v3/projects/458f8b87a0b84582a8ab8f17d39fad8d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:34.509276 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d4aa28-b4b7-46cc-8cde-9db1e7edd7c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=458f8b87a0b84582a8ab8f17d39fad8d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:34.511385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d4aa28-b4b7-46cc-8cde-9db1e7edd7c0 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:34.564752 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-88d4aa28-b4b7-46cc-8cde-9db1e7edd7c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '458f8b87a0b84582a8ab8f17d39fad8d', 'request_id': 'req-88d4aa28-b4b7-46cc-8cde-9db1e7edd7c0'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:34.573205 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-88d4aa28-b4b7-46cc-8cde-9db1e7edd7c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '458f8b87a0b84582a8ab8f17d39fad8d', 'request_id': 'req-88d4aa28-b4b7-46cc-8cde-9db1e7edd7c0'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:34.582480 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-88d4aa28-b4b7-46cc-8cde-9db1e7edd7c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '458f8b87a0b84582a8ab8f17d39fad8d', 'request_id': 'req-88d4aa28-b4b7-46cc-8cde-9db1e7edd7c0'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:34.597818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-88d4aa28-b4b7-46cc-8cde-9db1e7edd7c0 admin admin] The token cache is being invalidate because project 458f8b87a0b84582a8ab8f17d39fad8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:34.598023 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-88d4aa28-b4b7-46cc-8cde-9db1e7edd7c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:34.598685 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-88d4aa28-b4b7-46cc-8cde-9db1e7edd7c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:34.604521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1073/2148] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:34 2022] DELETE /identity/v3/projects/458f8b87a0b84582a8ab8f17d39fad8d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:34.607718 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb9aa2-4d58-417a-8b0c-ca46bf813284 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:34.611246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb9aa2-4d58-417a-8b0c-ca46bf813284 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:34.611246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb9aa2-4d58-417a-8b0c-ca46bf813284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:34.647563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-b3cb9aa2-4d58-417a-8b0c-ca46bf813284 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:34.649031 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b3cb9aa2-4d58-417a-8b0c-ca46bf813284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:34.740847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1076/2149] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:34.742504 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669313a-446b-45be-84ac-c143b33b910c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:34.742791 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669313a-446b-45be-84ac-c143b33b910c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:34.743068 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669313a-446b-45be-84ac-c143b33b910c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:34.762932 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-4669313a-446b-45be-84ac-c143b33b910c None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:34.766409 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-4669313a-446b-45be-84ac-c143b33b910c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:34.782662 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1074/2150] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:34.783835 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb9aa2-4d58-417a-8b0c-ca46bf813284 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:34.795927 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb9aa2-4d58-417a-8b0c-ca46bf813284 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:34.797392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669313a-446b-45be-84ac-c143b33b910c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:34.799509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-4669313a-446b-45be-84ac-c143b33b910c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:34.803624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62154adc-d69e-4c64-9630-2a99a27c96ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:34.804617 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62154adc-d69e-4c64-9630-2a99a27c96ff admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:34.805224 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62154adc-d69e-4c64-9630-2a99a27c96ff admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:34.805790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62154adc-d69e-4c64-9630-2a99a27c96ff admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:34.806789 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62154adc-d69e-4c64-9630-2a99a27c96ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:03:34.807495 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62154adc-d69e-4c64-9630-2a99a27c96ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:34.809645 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62154adc-d69e-4c64-9630-2a99a27c96ff admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:34.812992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669313a-446b-45be-84ac-c143b33b910c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:34.818949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b972f-6a81-49b1-ad1a-c0012034c3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:34.819926 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b972f-6a81-49b1-ad1a-c0012034c3a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:34.820348 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b972f-6a81-49b1-ad1a-c0012034c3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:34.820756 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b972f-6a81-49b1-ad1a-c0012034c3a9 admin admin] PATH_INFO: `/v3/users/77423f25c92243bb8ae21f420cabdf90` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:34.826214 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1077/2151] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:03:34.830638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b972f-6a81-49b1-ad1a-c0012034c3a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77423f25c92243bb8ae21f420cabdf90)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:34.831442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62154adc-d69e-4c64-9630-2a99a27c96ff admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:34.832980 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b972f-6a81-49b1-ad1a-c0012034c3a9 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:34.842861 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62154adc-d69e-4c64-9630-2a99a27c96ff admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:34.848167 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1c3ba-6521-4066-829b-f3a8edca736c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:34.848829 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1c3ba-6521-4066-829b-f3a8edca736c admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:34.849188 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1c3ba-6521-4066-829b-f3a8edca736c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:34.849560 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1c3ba-6521-4066-829b-f3a8edca736c admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:34.850362 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac1c3ba-6521-4066-829b-f3a8edca736c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:34.852514 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac1c3ba-6521-4066-829b-f3a8edca736c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:34.893132 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1078/2152] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:34 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:34.897640 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1c3ba-6521-4066-829b-f3a8edca736c admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:34.901711 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1a8b972f-6a81-49b1-ad1a-c0012034c3a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77423f25c92243bb8ae21f420cabdf90'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:34.921150 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1a8b972f-6a81-49b1-ad1a-c0012034c3a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77423f25c92243bb8ae21f420cabdf90'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:34.924288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1a8b972f-6a81-49b1-ad1a-c0012034c3a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77423f25c92243bb8ae21f420cabdf90'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:34.933852 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1a8b972f-6a81-49b1-ad1a-c0012034c3a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77423f25c92243bb8ae21f420cabdf90'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:34.934726 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1a8b972f-6a81-49b1-ad1a-c0012034c3a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77423f25c92243bb8ae21f420cabdf90'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:34.955297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1075/2153] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:34 2022] DELETE /identity/v3/users/77423f25c92243bb8ae21f420cabdf90 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:03:34.959848 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c00dbfe-9180-4c1b-b7af-efabebbee501 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:34.960379 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c00dbfe-9180-4c1b-b7af-efabebbee501 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:34.960625 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c00dbfe-9180-4c1b-b7af-efabebbee501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:34.980307 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-8c00dbfe-9180-4c1b-b7af-efabebbee501 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:34.981646 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8c00dbfe-9180-4c1b-b7af-efabebbee501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:34.990176 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1c3ba-6521-4066-829b-f3a8edca736c admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:35.059164 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1076/2154] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:03:35.068128 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b54a9-241a-4ed1-a8ef-607a6d4757b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:35.068804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b54a9-241a-4ed1-a8ef-607a6d4757b3 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:35.069083 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b54a9-241a-4ed1-a8ef-607a6d4757b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:35.069394 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b54a9-241a-4ed1-a8ef-607a6d4757b3 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:35.070488 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b54a9-241a-4ed1-a8ef-607a6d4757b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:35.071248 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c00dbfe-9180-4c1b-b7af-efabebbee501 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:35.072970 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8c00dbfe-9180-4c1b-b7af-efabebbee501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:35.075181 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b54a9-241a-4ed1-a8ef-607a6d4757b3 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:35.098243 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1079/2155] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:34 2022] POST /identity/v3/users => generated 288 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:35.102192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b54a9-241a-4ed1-a8ef-607a6d4757b3 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:35.104652 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-249b54a9-241a-4ed1-a8ef-607a6d4757b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:35.122847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b54a9-241a-4ed1-a8ef-607a6d4757b3 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:35.124050 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26197d-def2-4a4f-9057-2d6b9884e671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:35.125335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26197d-def2-4a4f-9057-2d6b9884e671 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:35.125614 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26197d-def2-4a4f-9057-2d6b9884e671 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:35.125880 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26197d-def2-4a4f-9057-2d6b9884e671 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:35.126645 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26197d-def2-4a4f-9057-2d6b9884e671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:35.128211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26197d-def2-4a4f-9057-2d6b9884e671 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:35.138833 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c00dbfe-9180-4c1b-b7af-efabebbee501 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:35.139753 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1080/2156] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:35.143790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26197d-def2-4a4f-9057-2d6b9884e671 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:35.146358 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b769b8-2772-4e0b-9a8f-b6aefaf4775a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:35.147058 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b769b8-2772-4e0b-9a8f-b6aefaf4775a service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:35.147448 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b769b8-2772-4e0b-9a8f-b6aefaf4775a service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:35.147745 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b769b8-2772-4e0b-9a8f-b6aefaf4775a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:35.155338 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26197d-def2-4a4f-9057-2d6b9884e671 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:35.160139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b769b8-2772-4e0b-9a8f-b6aefaf4775a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:35.160289 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492f89c-5ec7-4880-89ec-449fd03f7a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:35.160834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492f89c-5ec7-4880-89ec-449fd03f7a6d admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:35.161111 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492f89c-5ec7-4880-89ec-449fd03f7a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:35.161410 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492f89c-5ec7-4880-89ec-449fd03f7a6d admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:35.161594 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b769b8-2772-4e0b-9a8f-b6aefaf4775a service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:35.162149 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9492f89c-5ec7-4880-89ec-449fd03f7a6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:35.163758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9492f89c-5ec7-4880-89ec-449fd03f7a6d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:35.172618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1077/2157] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 23:03:35.178081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1081/2158] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:35.179279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b769b8-2772-4e0b-9a8f-b6aefaf4775a service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:35.192130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b769b8-2772-4e0b-9a8f-b6aefaf4775a service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:35.197289 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bb780-eb65-4c02-baab-24016154f96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:35.198495 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bb780-eb65-4c02-baab-24016154f96b admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:35.198608 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bb780-eb65-4c02-baab-24016154f96b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:35.198961 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bb780-eb65-4c02-baab-24016154f96b admin admin] PATH_INFO: `/v3/projects/fcbf3360a1654ddbbc5812f5ea706eb6/users/0f9a151d58eb4c8a8e99c1e35e6723f6/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:35.220579 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991bb780-eb65-4c02-baab-24016154f96b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcbf3360a1654ddbbc5812f5ea706eb6, user_id=0f9a151d58eb4c8a8e99c1e35e6723f6, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:35.224985 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991bb780-eb65-4c02-baab-24016154f96b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:35.244641 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1078/2159] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:35 2022] PUT /identity/v3/projects/fcbf3360a1654ddbbc5812f5ea706eb6/users/0f9a151d58eb4c8a8e99c1e35e6723f6/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:03:35.256006 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cffec-31b3-4a92-a391-2e8e014aae8f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:35.256721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cffec-31b3-4a92-a391-2e8e014aae8f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:35.257176 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cffec-31b3-4a92-a391-2e8e014aae8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:35.304057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c63cffec-31b3-4a92-a391-2e8e014aae8f None None] MFA Rules not processed for user `0f9a151d58eb4c8a8e99c1e35e6723f6`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:35.306095 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c63cffec-31b3-4a92-a391-2e8e014aae8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:35.444325 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1082/2160] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:35.457392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bb780-eb65-4c02-baab-24016154f96b admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:35.459195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-991bb780-eb65-4c02-baab-24016154f96b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:35.535902 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cffec-31b3-4a92-a391-2e8e014aae8f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:35.537737 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c63cffec-31b3-4a92-a391-2e8e014aae8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:35.549477 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bb780-eb65-4c02-baab-24016154f96b admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:35.549477 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72055f2b-fe8d-4033-be95-5369feb5ce03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:35.549477 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72055f2b-fe8d-4033-be95-5369feb5ce03 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:35.549477 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72055f2b-fe8d-4033-be95-5369feb5ce03 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:35.550640 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72055f2b-fe8d-4033-be95-5369feb5ce03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:35.558221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72055f2b-fe8d-4033-be95-5369feb5ce03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:35.559865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72055f2b-fe8d-4033-be95-5369feb5ce03 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:35.573192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1079/2161] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:35.616572 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cffec-31b3-4a92-a391-2e8e014aae8f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:35.625133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400d63b-179e-408c-a17e-16b1b1063ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:35.626081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400d63b-179e-408c-a17e-16b1b1063ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:35.626395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400d63b-179e-408c-a17e-16b1b1063ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:35.626663 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400d63b-179e-408c-a17e-16b1b1063ad1 admin admin] PATH_INFO: `/v3/projects/377be26fb207447ba6baad14693c76f1` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:35.628194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400d63b-179e-408c-a17e-16b1b1063ad1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=377be26fb207447ba6baad14693c76f1)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:35.634216 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400d63b-179e-408c-a17e-16b1b1063ad1 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:35.680350 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a400d63b-179e-408c-a17e-16b1b1063ad1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '377be26fb207447ba6baad14693c76f1', 'request_id': 'req-a400d63b-179e-408c-a17e-16b1b1063ad1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:35.689893 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a400d63b-179e-408c-a17e-16b1b1063ad1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '377be26fb207447ba6baad14693c76f1', 'request_id': 'req-a400d63b-179e-408c-a17e-16b1b1063ad1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:35.696597 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a400d63b-179e-408c-a17e-16b1b1063ad1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '377be26fb207447ba6baad14693c76f1', 'request_id': 'req-a400d63b-179e-408c-a17e-16b1b1063ad1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:35.711024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72055f2b-fe8d-4033-be95-5369feb5ce03 service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:35.712540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a400d63b-179e-408c-a17e-16b1b1063ad1 admin admin] The token cache is being invalidate because project 377be26fb207447ba6baad14693c76f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:35.712677 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-72055f2b-fe8d-4033-be95-5369feb5ce03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:35.712836 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a400d63b-179e-408c-a17e-16b1b1063ad1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:35.713524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a400d63b-179e-408c-a17e-16b1b1063ad1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:35.715259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1083/2162] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:35 2022] DELETE /identity/v3/projects/377be26fb207447ba6baad14693c76f1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:35.719350 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400d63b-179e-408c-a17e-16b1b1063ad1 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:35.720696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a400d63b-179e-408c-a17e-16b1b1063ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:35.727180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72055f2b-fe8d-4033-be95-5369feb5ce03 service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:35.788611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f718fd7-2907-4271-9dcb-70ee9cce6e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:35.789761 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f718fd7-2907-4271-9dcb-70ee9cce6e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:35.790044 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f718fd7-2907-4271-9dcb-70ee9cce6e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:35.790368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f718fd7-2907-4271-9dcb-70ee9cce6e1d admin admin] PATH_INFO: `/v3/users/502516b956344e4191bab16c4338698a` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:35.791585 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f718fd7-2907-4271-9dcb-70ee9cce6e1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=502516b956344e4191bab16c4338698a)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:35.792278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400d63b-179e-408c-a17e-16b1b1063ad1 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:35.793509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f718fd7-2907-4271-9dcb-70ee9cce6e1d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:35.797250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ae803-526e-403f-928f-7864ba214694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:35.798752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ae803-526e-403f-928f-7864ba214694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:35.798752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ae803-526e-403f-928f-7864ba214694 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:35.798977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ae803-526e-403f-928f-7864ba214694 admin admin] PATH_INFO: `/v3/users/d030e4506aeb40d9aaa221a625958531` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:35.800702 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ae803-526e-403f-928f-7864ba214694 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d030e4506aeb40d9aaa221a625958531)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:35.802941 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ae803-526e-403f-928f-7864ba214694 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:35.881335 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3f718fd7-2907-4271-9dcb-70ee9cce6e1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '502516b956344e4191bab16c4338698a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:35.890862 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3f718fd7-2907-4271-9dcb-70ee9cce6e1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '502516b956344e4191bab16c4338698a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:35.891649 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3f718fd7-2907-4271-9dcb-70ee9cce6e1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '502516b956344e4191bab16c4338698a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:35.892250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-af4ae803-526e-403f-928f-7864ba214694 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd030e4506aeb40d9aaa221a625958531'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:35.900776 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3f718fd7-2907-4271-9dcb-70ee9cce6e1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '502516b956344e4191bab16c4338698a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:35.902110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3f718fd7-2907-4271-9dcb-70ee9cce6e1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '502516b956344e4191bab16c4338698a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:35.912255 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-af4ae803-526e-403f-928f-7864ba214694 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd030e4506aeb40d9aaa221a625958531'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:35.921555 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-af4ae803-526e-403f-928f-7864ba214694 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd030e4506aeb40d9aaa221a625958531'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:35.922317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-af4ae803-526e-403f-928f-7864ba214694 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd030e4506aeb40d9aaa221a625958531'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:35.926167 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1080/2163] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:35 2022] DELETE /identity/v3/users/502516b956344e4191bab16c4338698a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:35.930623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd93d35-f401-430c-8f91-a98e4361771c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:35.930972 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd93d35-f401-430c-8f91-a98e4361771c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:35.931254 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd93d35-f401-430c-8f91-a98e4361771c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:35.935415 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-af4ae803-526e-403f-928f-7864ba214694 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd030e4506aeb40d9aaa221a625958531'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:35.942901 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1084/2164] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:35 2022] DELETE /identity/v3/users/d030e4506aeb40d9aaa221a625958531 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:03:35.957556 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-2bd93d35-f401-430c-8f91-a98e4361771c None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:35.958810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2bd93d35-f401-430c-8f91-a98e4361771c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:36.021641 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1081/2165] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:03:36.031318 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ae803-526e-403f-928f-7864ba214694 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:36.032736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-af4ae803-526e-403f-928f-7864ba214694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:36.090442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ae803-526e-403f-928f-7864ba214694 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:36.094486 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db20596e-5999-4683-9a09-7a48a6d1ce06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:36.095258 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20596e-5999-4683-9a09-7a48a6d1ce06 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:36.095549 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20596e-5999-4683-9a09-7a48a6d1ce06 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:36.095824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20596e-5999-4683-9a09-7a48a6d1ce06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:36.105795 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db20596e-5999-4683-9a09-7a48a6d1ce06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:36.107521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db20596e-5999-4683-9a09-7a48a6d1ce06 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:36.117496 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1085/2166] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:36.291083 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd93d35-f401-430c-8f91-a98e4361771c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:36.293382 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2bd93d35-f401-430c-8f91-a98e4361771c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:36.309336 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd93d35-f401-430c-8f91-a98e4361771c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:36.315206 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5935154-52b9-4ff2-b48b-2cafffe95b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:36.316069 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5935154-52b9-4ff2-b48b-2cafffe95b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:36.316806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5935154-52b9-4ff2-b48b-2cafffe95b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:36.317092 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5935154-52b9-4ff2-b48b-2cafffe95b53 admin admin] PATH_INFO: `/v3/projects/9ebe6358b6ab4e58a9fd05df062dbe98` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:36.319053 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5935154-52b9-4ff2-b48b-2cafffe95b53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ebe6358b6ab4e58a9fd05df062dbe98)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:36.321306 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5935154-52b9-4ff2-b48b-2cafffe95b53 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:36.401640 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e5935154-52b9-4ff2-b48b-2cafffe95b53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ebe6358b6ab4e58a9fd05df062dbe98', 'request_id': 'req-e5935154-52b9-4ff2-b48b-2cafffe95b53'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:36.412958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e5935154-52b9-4ff2-b48b-2cafffe95b53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ebe6358b6ab4e58a9fd05df062dbe98', 'request_id': 'req-e5935154-52b9-4ff2-b48b-2cafffe95b53'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:36.419556 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e5935154-52b9-4ff2-b48b-2cafffe95b53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ebe6358b6ab4e58a9fd05df062dbe98', 'request_id': 'req-e5935154-52b9-4ff2-b48b-2cafffe95b53'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:36.439058 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e5935154-52b9-4ff2-b48b-2cafffe95b53 admin admin] The token cache is being invalidate because project 9ebe6358b6ab4e58a9fd05df062dbe98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:36.439271 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e5935154-52b9-4ff2-b48b-2cafffe95b53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:36.439936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e5935154-52b9-4ff2-b48b-2cafffe95b53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:36.445184 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1082/2167] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:36 2022] DELETE /identity/v3/projects/9ebe6358b6ab4e58a9fd05df062dbe98 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:36.447594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfb207-0d90-4ca3-9d7e-284ab8b781f0 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:36.447848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfb207-0d90-4ca3-9d7e-284ab8b781f0 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:36.448128 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfb207-0d90-4ca3-9d7e-284ab8b781f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:36.473015 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f4cfb207-0d90-4ca3-9d7e-284ab8b781f0 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:36.474342 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f4cfb207-0d90-4ca3-9d7e-284ab8b781f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:36.515978 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5935154-52b9-4ff2-b48b-2cafffe95b53 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:36.521292 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e5935154-52b9-4ff2-b48b-2cafffe95b53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:36.543546 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1086/2168] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:36.552390 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca014bf-84e5-4942-94bf-6d8850860f86 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:36.552390 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca014bf-84e5-4942-94bf-6d8850860f86 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:36.552390 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca014bf-84e5-4942-94bf-6d8850860f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:36.591807 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3ca014bf-84e5-4942-94bf-6d8850860f86 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:36.593035 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3ca014bf-84e5-4942-94bf-6d8850860f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:36.602968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1087/2169] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:03:36.606823 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca014bf-84e5-4942-94bf-6d8850860f86 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:36.611206 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5935154-52b9-4ff2-b48b-2cafffe95b53 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:36.616088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb363c5e-ba64-48c7-8227-fea6910e0187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:36.617041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb363c5e-ba64-48c7-8227-fea6910e0187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:36.617391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb363c5e-ba64-48c7-8227-fea6910e0187 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:36.617732 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb363c5e-ba64-48c7-8227-fea6910e0187 admin admin] PATH_INFO: `/v3/projects/c0c9afa417d540428c522e6658b7f90e` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:36.618775 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca014bf-84e5-4942-94bf-6d8850860f86 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:36.619195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb363c5e-ba64-48c7-8227-fea6910e0187 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0c9afa417d540428c522e6658b7f90e)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:36.621147 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb363c5e-ba64-48c7-8227-fea6910e0187 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:36.629466 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed29be-4575-4887-9917-13122ee71b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:36.630250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed29be-4575-4887-9917-13122ee71b83 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:36.630909 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed29be-4575-4887-9917-13122ee71b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:36.631199 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed29be-4575-4887-9917-13122ee71b83 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:36.632069 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed29be-4575-4887-9917-13122ee71b83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:03:36.632790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed29be-4575-4887-9917-13122ee71b83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:36.639050 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed29be-4575-4887-9917-13122ee71b83 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:36.650336 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1088/2170] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:03:36.654403 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed29be-4575-4887-9917-13122ee71b83 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:36.666903 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed29be-4575-4887-9917-13122ee71b83 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:36.672937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0267f8b-2833-43c9-9ce0-1b904d9e4c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:36.673739 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0267f8b-2833-43c9-9ce0-1b904d9e4c2c admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:36.674030 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0267f8b-2833-43c9-9ce0-1b904d9e4c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:36.674333 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0267f8b-2833-43c9-9ce0-1b904d9e4c2c admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:36.675022 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0267f8b-2833-43c9-9ce0-1b904d9e4c2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:36.676929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0267f8b-2833-43c9-9ce0-1b904d9e4c2c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:36.692503 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eb363c5e-ba64-48c7-8227-fea6910e0187 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0c9afa417d540428c522e6658b7f90e', 'request_id': 'req-eb363c5e-ba64-48c7-8227-fea6910e0187'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:36.699505 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eb363c5e-ba64-48c7-8227-fea6910e0187 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0c9afa417d540428c522e6658b7f90e', 'request_id': 'req-eb363c5e-ba64-48c7-8227-fea6910e0187'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:36.705083 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eb363c5e-ba64-48c7-8227-fea6910e0187 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0c9afa417d540428c522e6658b7f90e', 'request_id': 'req-eb363c5e-ba64-48c7-8227-fea6910e0187'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:36.706480 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1089/2171] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:36 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:36.710549 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0267f8b-2833-43c9-9ce0-1b904d9e4c2c admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:36.718932 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eb363c5e-ba64-48c7-8227-fea6910e0187 admin admin] The token cache is being invalidate because project c0c9afa417d540428c522e6658b7f90e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:36.719429 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eb363c5e-ba64-48c7-8227-fea6910e0187 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:36.720139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eb363c5e-ba64-48c7-8227-fea6910e0187 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:36.721967 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1083/2172] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:36 2022] DELETE /identity/v3/projects/c0c9afa417d540428c522e6658b7f90e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:36.793377 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0267f8b-2833-43c9-9ce0-1b904d9e4c2c admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:36.799633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165b267-762f-4727-a997-f1fce5ff3b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:36.800714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165b267-762f-4727-a997-f1fce5ff3b62 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:36.801192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165b267-762f-4727-a997-f1fce5ff3b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:36.801658 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165b267-762f-4727-a997-f1fce5ff3b62 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:36.802674 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165b267-762f-4727-a997-f1fce5ff3b62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:36.805048 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165b267-762f-4727-a997-f1fce5ff3b62 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:36.830487 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1090/2173] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:36 2022] POST /identity/v3/users => generated 283 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:03:36.834406 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb363c5e-ba64-48c7-8227-fea6910e0187 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:36.835856 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-eb363c5e-ba64-48c7-8227-fea6910e0187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:36.920254 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb363c5e-ba64-48c7-8227-fea6910e0187 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:36.926670 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80537fd-b73b-4b6a-b671-e8a0d3332088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:36.927729 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80537fd-b73b-4b6a-b671-e8a0d3332088 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:36.928111 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80537fd-b73b-4b6a-b671-e8a0d3332088 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:36.928493 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80537fd-b73b-4b6a-b671-e8a0d3332088 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:36.929539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80537fd-b73b-4b6a-b671-e8a0d3332088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:36.931663 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80537fd-b73b-4b6a-b671-e8a0d3332088 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:36.948825 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1084/2174] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:36.949803 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165b267-762f-4727-a997-f1fce5ff3b62 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:36.962453 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165b267-762f-4727-a997-f1fce5ff3b62 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:36.968838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ba166-8964-43d1-9f2a-fc592ea36f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:36.969774 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ba166-8964-43d1-9f2a-fc592ea36f2a admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:36.970155 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ba166-8964-43d1-9f2a-fc592ea36f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:36.970612 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ba166-8964-43d1-9f2a-fc592ea36f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:36.978228 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ba166-8964-43d1-9f2a-fc592ea36f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:36.980217 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ba166-8964-43d1-9f2a-fc592ea36f2a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:36.998420 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503de7c-dfa2-45db-b485-009d596975b5 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:36.998712 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503de7c-dfa2-45db-b485-009d596975b5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:36.998995 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503de7c-dfa2-45db-b485-009d596975b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:37.002937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1091/2175] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:37.004743 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ba166-8964-43d1-9f2a-fc592ea36f2a admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:37.018559 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ba166-8964-43d1-9f2a-fc592ea36f2a admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:37.019679 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-4503de7c-dfa2-45db-b485-009d596975b5 None None] MFA Rules not processed for user `0f9a151d58eb4c8a8e99c1e35e6723f6`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:37.020901 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-4503de7c-dfa2-45db-b485-009d596975b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:37.024053 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a534d-af68-4c34-a2f1-d45c8267adad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:37.025035 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a534d-af68-4c34-a2f1-d45c8267adad admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:37.025376 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a534d-af68-4c34-a2f1-d45c8267adad admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:37.025726 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a534d-af68-4c34-a2f1-d45c8267adad admin admin] PATH_INFO: `/v3/projects/62d72bb5612346a68ed3d6f4a524f2a2/users/bb3b39bfddeb4e21a3fe4e31cb9b987e/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:37.042513 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a534d-af68-4c34-a2f1-d45c8267adad admin admin] RBAC: Authorizing `identity:create_grant(project_id=62d72bb5612346a68ed3d6f4a524f2a2, user_id=bb3b39bfddeb4e21a3fe4e31cb9b987e, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:37.047354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a534d-af68-4c34-a2f1-d45c8267adad admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:37.068818 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1092/2176] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:37 2022] PUT /identity/v3/projects/62d72bb5612346a68ed3d6f4a524f2a2/users/bb3b39bfddeb4e21a3fe4e31cb9b987e/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:37.080236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea546da9-1cff-4f1a-892e-77349a7eb52e None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:37.080528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea546da9-1cff-4f1a-892e-77349a7eb52e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:37.080803 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea546da9-1cff-4f1a-892e-77349a7eb52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:37.106945 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1085/2177] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:03:37.111290 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-ea546da9-1cff-4f1a-892e-77349a7eb52e None None] MFA Rules not processed for user `bb3b39bfddeb4e21a3fe4e31cb9b987e`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:37.112597 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ea546da9-1cff-4f1a-892e-77349a7eb52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:37.151036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4503de7c-dfa2-45db-b485-009d596975b5 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:37.152604 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-4503de7c-dfa2-45db-b485-009d596975b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:37.167303 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4503de7c-dfa2-45db-b485-009d596975b5 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:37.184978 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1093/2178] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:37 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:03:37.189553 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea546da9-1cff-4f1a-892e-77349a7eb52e None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:37.236940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453cbfec-5c79-470f-a902-ff7a15e0a7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:37.237808 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cbfec-5c79-470f-a902-ff7a15e0a7dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:37.238086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cbfec-5c79-470f-a902-ff7a15e0a7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:37.238384 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cbfec-5c79-470f-a902-ff7a15e0a7dd admin admin] PATH_INFO: `/v3/users/90018602e07f4a4ca5f46c5c85bc818e` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:37.239572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453cbfec-5c79-470f-a902-ff7a15e0a7dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=90018602e07f4a4ca5f46c5c85bc818e)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:37.241485 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453cbfec-5c79-470f-a902-ff7a15e0a7dd admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:37.263624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea546da9-1cff-4f1a-892e-77349a7eb52e None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:37.268506 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e17c8-2801-41cc-9d11-242e1a3577df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:37.269180 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e17c8-2801-41cc-9d11-242e1a3577df admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:37.269491 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e17c8-2801-41cc-9d11-242e1a3577df admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:37.269759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e17c8-2801-41cc-9d11-242e1a3577df admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:37.270509 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191e17c8-2801-41cc-9d11-242e1a3577df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:37.272486 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191e17c8-2801-41cc-9d11-242e1a3577df admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:37.285942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-453cbfec-5c79-470f-a902-ff7a15e0a7dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90018602e07f4a4ca5f46c5c85bc818e'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:37.292973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-453cbfec-5c79-470f-a902-ff7a15e0a7dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90018602e07f4a4ca5f46c5c85bc818e'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:37.293680 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-453cbfec-5c79-470f-a902-ff7a15e0a7dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90018602e07f4a4ca5f46c5c85bc818e'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:37.298371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1094/2179] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:37 2022] POST /identity/v3/projects => generated 367 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:03:37.302385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e17c8-2801-41cc-9d11-242e1a3577df admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:37.304062 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-191e17c8-2801-41cc-9d11-242e1a3577df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:37.305823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-453cbfec-5c79-470f-a902-ff7a15e0a7dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90018602e07f4a4ca5f46c5c85bc818e'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:37.306500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-453cbfec-5c79-470f-a902-ff7a15e0a7dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90018602e07f4a4ca5f46c5c85bc818e'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:37.323458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1086/2180] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:37 2022] DELETE /identity/v3/users/90018602e07f4a4ca5f46c5c85bc818e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:03:37.366271 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e17c8-2801-41cc-9d11-242e1a3577df admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:37.420433 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771896f-361d-4b4c-a929-84bbcd5bf45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:37.421121 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771896f-361d-4b4c-a929-84bbcd5bf45a admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:37.421434 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771896f-361d-4b4c-a929-84bbcd5bf45a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:37.421709 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771896f-361d-4b4c-a929-84bbcd5bf45a admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:37.422442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771896f-361d-4b4c-a929-84bbcd5bf45a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:37.424370 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771896f-361d-4b4c-a929-84bbcd5bf45a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:37.449650 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1095/2181] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:37 2022] POST /identity/v3/users => generated 290 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:03:37.450317 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453cbfec-5c79-470f-a902-ff7a15e0a7dd admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:37.461645 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453cbfec-5c79-470f-a902-ff7a15e0a7dd admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:37.474950 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5b831-1efd-4bc5-9daa-26832eb09446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:37.475695 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5b831-1efd-4bc5-9daa-26832eb09446 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:37.475954 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5b831-1efd-4bc5-9daa-26832eb09446 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:37.476192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5b831-1efd-4bc5-9daa-26832eb09446 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:37.476928 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5b831-1efd-4bc5-9daa-26832eb09446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:37.479219 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5b831-1efd-4bc5-9daa-26832eb09446 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:37.490225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1087/2182] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:37.494701 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771896f-361d-4b4c-a929-84bbcd5bf45a admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:37.510364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771896f-361d-4b4c-a929-84bbcd5bf45a admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:37.516138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835438e1-b918-49b6-b03a-ad66c0391455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:37.517112 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835438e1-b918-49b6-b03a-ad66c0391455 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:37.517416 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835438e1-b918-49b6-b03a-ad66c0391455 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:37.517679 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835438e1-b918-49b6-b03a-ad66c0391455 admin admin] PATH_INFO: `/v3/projects/465639098487433bae134893633f98e3/users/b5fa58b1e92f47b5b22eb9fecd4f37cc/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:37.539414 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5b831-1efd-4bc5-9daa-26832eb09446 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:37.540022 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835438e1-b918-49b6-b03a-ad66c0391455 admin admin] RBAC: Authorizing `identity:create_grant(project_id=465639098487433bae134893633f98e3, user_id=b5fa58b1e92f47b5b22eb9fecd4f37cc, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:37.541317 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-61d5b831-1efd-4bc5-9daa-26832eb09446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:37.543027 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835438e1-b918-49b6-b03a-ad66c0391455 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:37.564712 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1096/2183] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:37 2022] PUT /identity/v3/projects/465639098487433bae134893633f98e3/users/b5fa58b1e92f47b5b22eb9fecd4f37cc/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:37.571818 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835438e1-b918-49b6-b03a-ad66c0391455 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:37.574623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5b831-1efd-4bc5-9daa-26832eb09446 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:37.634336 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835438e1-b918-49b6-b03a-ad66c0391455 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:37.639780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc0f00-19af-4bb6-8993-d6fa0a41db5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:37.641310 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc0f00-19af-4bb6-8993-d6fa0a41db5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:37.641636 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc0f00-19af-4bb6-8993-d6fa0a41db5d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:37.641864 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc0f00-19af-4bb6-8993-d6fa0a41db5d admin admin] PATH_INFO: `/v3/projects/0805f702593a4c379db7cdeec818f94b` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:37.643743 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0ed65-3f60-45ef-8035-1c725c55eed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:37.644910 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc0f00-19af-4bb6-8993-d6fa0a41db5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0805f702593a4c379db7cdeec818f94b)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:37.645696 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0ed65-3f60-45ef-8035-1c725c55eed4 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:37.645981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0ed65-3f60-45ef-8035-1c725c55eed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:37.646294 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0ed65-3f60-45ef-8035-1c725c55eed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:37.646995 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc0f00-19af-4bb6-8993-d6fa0a41db5d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:37.647118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df0ed65-3f60-45ef-8035-1c725c55eed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:37.648600 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df0ed65-3f60-45ef-8035-1c725c55eed4 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:37.662314 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1097/2184] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:37.663304 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0ed65-3f60-45ef-8035-1c725c55eed4 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:37.672681 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0ed65-3f60-45ef-8035-1c725c55eed4 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:37.677924 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f9bbb-c9df-499a-8e17-b2dafb447f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:37.678819 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f9bbb-c9df-499a-8e17-b2dafb447f9a admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:37.679092 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f9bbb-c9df-499a-8e17-b2dafb447f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:37.679394 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f9bbb-c9df-499a-8e17-b2dafb447f9a admin admin] PATH_INFO: `/v3/projects/465639098487433bae134893633f98e3/users/b5fa58b1e92f47b5b22eb9fecd4f37cc/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:37.681334 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f9bbb-c9df-499a-8e17-b2dafb447f9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=465639098487433bae134893633f98e3, user_id=b5fa58b1e92f47b5b22eb9fecd4f37cc, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:37.683993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f9bbb-c9df-499a-8e17-b2dafb447f9a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:37.692711 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-bcdc0f00-19af-4bb6-8993-d6fa0a41db5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0805f702593a4c379db7cdeec818f94b', 'request_id': 'req-bcdc0f00-19af-4bb6-8993-d6fa0a41db5d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:37.695260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1098/2185] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:37 2022] PUT /identity/v3/projects/465639098487433bae134893633f98e3/users/b5fa58b1e92f47b5b22eb9fecd4f37cc/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:37.698787 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-bcdc0f00-19af-4bb6-8993-d6fa0a41db5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0805f702593a4c379db7cdeec818f94b', 'request_id': 'req-bcdc0f00-19af-4bb6-8993-d6fa0a41db5d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:37.704831 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-bcdc0f00-19af-4bb6-8993-d6fa0a41db5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0805f702593a4c379db7cdeec818f94b', 'request_id': 'req-bcdc0f00-19af-4bb6-8993-d6fa0a41db5d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:37.706056 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917901c-0553-4cf6-b3a2-abfb28b3fc3e None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:37.706408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917901c-0553-4cf6-b3a2-abfb28b3fc3e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:37.706693 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917901c-0553-4cf6-b3a2-abfb28b3fc3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:37.718093 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-bcdc0f00-19af-4bb6-8993-d6fa0a41db5d admin admin] The token cache is being invalidate because project 0805f702593a4c379db7cdeec818f94b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:37.719061 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-bcdc0f00-19af-4bb6-8993-d6fa0a41db5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:37.722050 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-bcdc0f00-19af-4bb6-8993-d6fa0a41db5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:37.723905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1088/2186] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:37 2022] DELETE /identity/v3/projects/0805f702593a4c379db7cdeec818f94b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:03:37.728800 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f9d90-e3b4-4f82-ba3d-28524b6fe3d8 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:37.729092 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f9d90-e3b4-4f82-ba3d-28524b6fe3d8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:37.729374 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f9d90-e3b4-4f82-ba3d-28524b6fe3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:37.740949 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c917901c-0553-4cf6-b3a2-abfb28b3fc3e None None] MFA Rules not processed for user `b5fa58b1e92f47b5b22eb9fecd4f37cc`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:37.742433 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c917901c-0553-4cf6-b3a2-abfb28b3fc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:37.748569 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-7e5f9d90-e3b4-4f82-ba3d-28524b6fe3d8 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:37.750392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7e5f9d90-e3b4-4f82-ba3d-28524b6fe3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:37.807524 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1089/2187] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:03:37.816140 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1099/2188] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:37 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:37.816919 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec66f8-8741-4059-910a-9b7028ca3c6e None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:37.817216 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec66f8-8741-4059-910a-9b7028ca3c6e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:37.817495 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec66f8-8741-4059-910a-9b7028ca3c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:37.825585 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917901c-0553-4cf6-b3a2-abfb28b3fc3e None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:37.827042 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c917901c-0553-4cf6-b3a2-abfb28b3fc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:37.836424 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e1ec66f8-8741-4059-910a-9b7028ca3c6e None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:37.837757 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e1ec66f8-8741-4059-910a-9b7028ca3c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:37.845684 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1090/2189] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:03:37.849781 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec66f8-8741-4059-910a-9b7028ca3c6e None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:37.859848 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec66f8-8741-4059-910a-9b7028ca3c6e None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:37.869527 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864236eb-272b-44d3-8e43-acf2e0450daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:37.870237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864236eb-272b-44d3-8e43-acf2e0450daf admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:37.870514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864236eb-272b-44d3-8e43-acf2e0450daf admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:37.870780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864236eb-272b-44d3-8e43-acf2e0450daf admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:37.871506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864236eb-272b-44d3-8e43-acf2e0450daf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:03:37.871802 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864236eb-272b-44d3-8e43-acf2e0450daf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:37.873486 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864236eb-272b-44d3-8e43-acf2e0450daf admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:37.885577 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1091/2190] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 23:03:37.886592 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917901c-0553-4cf6-b3a2-abfb28b3fc3e None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:37.889795 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864236eb-272b-44d3-8e43-acf2e0450daf admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:37.891143 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfea8c-8961-4800-b748-ffc2822d00b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:37.891808 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfea8c-8961-4800-b748-ffc2822d00b3 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:37.892086 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfea8c-8961-4800-b748-ffc2822d00b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:37.892356 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfea8c-8961-4800-b748-ffc2822d00b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:37.899539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864236eb-272b-44d3-8e43-acf2e0450daf admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:37.901324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dfea8c-8961-4800-b748-ffc2822d00b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:37.902777 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dfea8c-8961-4800-b748-ffc2822d00b3 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:37.904067 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ce084-ec6f-44b6-b28f-2b549f5bb5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:37.904682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ce084-ec6f-44b6-b28f-2b549f5bb5ae admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:37.904954 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ce084-ec6f-44b6-b28f-2b549f5bb5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:37.905222 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ce084-ec6f-44b6-b28f-2b549f5bb5ae admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:37.905922 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ce084-ec6f-44b6-b28f-2b549f5bb5ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:37.907802 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ce084-ec6f-44b6-b28f-2b549f5bb5ae admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:37.912008 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1100/2191] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:37 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 23:03:37.935189 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1092/2192] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:37 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:03:37.939722 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfea8c-8961-4800-b748-ffc2822d00b3 service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:37.950030 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfea8c-8961-4800-b748-ffc2822d00b3 service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:37.995356 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ce084-ec6f-44b6-b28f-2b549f5bb5ae admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:38.007332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a037d-6519-4f51-bd9f-925b7be76121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:38.007999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a037d-6519-4f51-bd9f-925b7be76121 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:38.008283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a037d-6519-4f51-bd9f-925b7be76121 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:38.008565 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a037d-6519-4f51-bd9f-925b7be76121 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:38.009261 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a037d-6519-4f51-bd9f-925b7be76121 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:38.011211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a037d-6519-4f51-bd9f-925b7be76121 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:38.043152 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1101/2193] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:37 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:38.044722 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a037d-6519-4f51-bd9f-925b7be76121 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:38.058964 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a037d-6519-4f51-bd9f-925b7be76121 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:38.059180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ce084-ec6f-44b6-b28f-2b549f5bb5ae admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:38.065105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff20aa5-c8a4-4ff2-a051-5da84a6a4615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:38.066024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff20aa5-c8a4-4ff2-a051-5da84a6a4615 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:38.066411 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff20aa5-c8a4-4ff2-a051-5da84a6a4615 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:38.066915 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff20aa5-c8a4-4ff2-a051-5da84a6a4615 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:38.068030 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff20aa5-c8a4-4ff2-a051-5da84a6a4615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:38.068507 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b110e0d7-ca99-443d-8587-938d5b84ab29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:38.069213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110e0d7-ca99-443d-8587-938d5b84ab29 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:38.069511 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110e0d7-ca99-443d-8587-938d5b84ab29 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:38.069788 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110e0d7-ca99-443d-8587-938d5b84ab29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:38.069981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff20aa5-c8a4-4ff2-a051-5da84a6a4615 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:38.074159 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b110e0d7-ca99-443d-8587-938d5b84ab29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:38.081681 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1102/2194] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:38.085543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff20aa5-c8a4-4ff2-a051-5da84a6a4615 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:38.095158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff20aa5-c8a4-4ff2-a051-5da84a6a4615 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:38.099800 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240810b-ced3-4fad-ae5a-00cff04f2969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:38.100437 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240810b-ced3-4fad-ae5a-00cff04f2969 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:38.100718 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240810b-ced3-4fad-ae5a-00cff04f2969 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:38.100992 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240810b-ced3-4fad-ae5a-00cff04f2969 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:38.101693 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240810b-ced3-4fad-ae5a-00cff04f2969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:38.103651 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240810b-ced3-4fad-ae5a-00cff04f2969 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:38.118501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1103/2195] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:38.119811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240810b-ced3-4fad-ae5a-00cff04f2969 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:38.130523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240810b-ced3-4fad-ae5a-00cff04f2969 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:38.135240 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b110e0d7-ca99-443d-8587-938d5b84ab29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:38.135992 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc57ed-39c8-4408-98dd-53df683adda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:38.136677 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b110e0d7-ca99-443d-8587-938d5b84ab29 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:38.137041 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc57ed-39c8-4408-98dd-53df683adda6 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:38.137395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc57ed-39c8-4408-98dd-53df683adda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:38.137740 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc57ed-39c8-4408-98dd-53df683adda6 admin admin] PATH_INFO: `/v3/projects/0f38af4b388e49958aa17250c181b203/users/c950754974d748519118d974492f15ce/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:38.146076 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1093/2196] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:37 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 23:03:38.153006 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc57ed-39c8-4408-98dd-53df683adda6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f38af4b388e49958aa17250c181b203, user_id=c950754974d748519118d974492f15ce, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:38.155810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc57ed-39c8-4408-98dd-53df683adda6 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:38.172775 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1104/2197] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:38 2022] PUT /identity/v3/projects/0f38af4b388e49958aa17250c181b203/users/c950754974d748519118d974492f15ce/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:03:38.178931 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d9c3f-adcc-453d-a012-8f0181ca900e None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:38.179173 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d9c3f-adcc-453d-a012-8f0181ca900e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:38.179455 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d9c3f-adcc-453d-a012-8f0181ca900e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:38.203905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-992d9c3f-adcc-453d-a012-8f0181ca900e None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:38.205242 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-992d9c3f-adcc-453d-a012-8f0181ca900e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:38.278167 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1094/2198] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:38.287212 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc57ed-39c8-4408-98dd-53df683adda6 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:38.289137 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9edc57ed-39c8-4408-98dd-53df683adda6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:38.351464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc57ed-39c8-4408-98dd-53df683adda6 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:38.356020 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b9895-70a9-4149-8872-33a96a0f3259 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:38.356801 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b9895-70a9-4149-8872-33a96a0f3259 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:38.357435 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b9895-70a9-4149-8872-33a96a0f3259 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:38.357848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b9895-70a9-4149-8872-33a96a0f3259 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:38.373027 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b9895-70a9-4149-8872-33a96a0f3259 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:38.375419 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b9895-70a9-4149-8872-33a96a0f3259 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:38.387711 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1105/2199] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:43.674208 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d9c3f-adcc-453d-a012-8f0181ca900e None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:43.677798 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-992d9c3f-adcc-453d-a012-8f0181ca900e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:43.705855 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d9c3f-adcc-453d-a012-8f0181ca900e None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:43.717586 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d8e05-0303-48d1-a09a-b11ef0384d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:43.720280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d8e05-0303-48d1-a09a-b11ef0384d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:43.720819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d8e05-0303-48d1-a09a-b11ef0384d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:43.722624 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d8e05-0303-48d1-a09a-b11ef0384d14 admin admin] PATH_INFO: `/v3/users/bb3b39bfddeb4e21a3fe4e31cb9b987e` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:43.727021 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d8e05-0303-48d1-a09a-b11ef0384d14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb3b39bfddeb4e21a3fe4e31cb9b987e)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:43.729466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d8e05-0303-48d1-a09a-b11ef0384d14 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:43.859087 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8e0d8e05-0303-48d1-a09a-b11ef0384d14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb3b39bfddeb4e21a3fe4e31cb9b987e'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:43.866946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8e0d8e05-0303-48d1-a09a-b11ef0384d14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb3b39bfddeb4e21a3fe4e31cb9b987e'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:43.867871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8e0d8e05-0303-48d1-a09a-b11ef0384d14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb3b39bfddeb4e21a3fe4e31cb9b987e'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:43.879751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8e0d8e05-0303-48d1-a09a-b11ef0384d14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb3b39bfddeb4e21a3fe4e31cb9b987e'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:43.884705 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8e0d8e05-0303-48d1-a09a-b11ef0384d14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb3b39bfddeb4e21a3fe4e31cb9b987e'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:43.915438 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1095/2200] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:43 2022] DELETE /identity/v3/users/bb3b39bfddeb4e21a3fe4e31cb9b987e => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:43.917116 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8335c-728c-470b-9645-e9b9546a5dda None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:43.917559 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8335c-728c-470b-9645-e9b9546a5dda None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:43.917652 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8335c-728c-470b-9645-e9b9546a5dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:43.937574 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-7bd8335c-728c-470b-9645-e9b9546a5dda None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:43.938839 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7bd8335c-728c-470b-9645-e9b9546a5dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:43.961163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cd25e-cf00-4936-b709-62ae770d7cd5 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:43.961518 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cd25e-cf00-4936-b709-62ae770d7cd5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:43.961868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cd25e-cf00-4936-b709-62ae770d7cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:44.009527 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ad9cd25e-cf00-4936-b709-62ae770d7cd5 None None] MFA Rules not processed for user `c950754974d748519118d974492f15ce`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:44.012667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ad9cd25e-cf00-4936-b709-62ae770d7cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:44.051848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1106/2201] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:44.074228 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd8335c-728c-470b-9645-e9b9546a5dda None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:44.075798 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7bd8335c-728c-470b-9645-e9b9546a5dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:44.109642 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1096/2202] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:44.123260 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9cd25e-cf00-4936-b709-62ae770d7cd5 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:44.124844 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ad9cd25e-cf00-4936-b709-62ae770d7cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:44.138785 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd8335c-728c-470b-9645-e9b9546a5dda None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:44.148662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278e1d4-558c-442f-a716-cecef21b1cd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:44.153642 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278e1d4-558c-442f-a716-cecef21b1cd1 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:44.154236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278e1d4-558c-442f-a716-cecef21b1cd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:44.155163 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278e1d4-558c-442f-a716-cecef21b1cd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:44.180090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278e1d4-558c-442f-a716-cecef21b1cd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:44.182161 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278e1d4-558c-442f-a716-cecef21b1cd1 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:44.196827 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1107/2203] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 23:03:44.216629 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9cd25e-cf00-4936-b709-62ae770d7cd5 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:44.223224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6db71-44dc-4c53-a4cf-efcc6e7ff7db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:44.223960 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6db71-44dc-4c53-a4cf-efcc6e7ff7db service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:44.224340 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6db71-44dc-4c53-a4cf-efcc6e7ff7db service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:44.224685 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6db71-44dc-4c53-a4cf-efcc6e7ff7db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:44.234894 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6db71-44dc-4c53-a4cf-efcc6e7ff7db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:44.236599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6db71-44dc-4c53-a4cf-efcc6e7ff7db service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:44.246297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1097/2204] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 23:03:44.936811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278e1d4-558c-442f-a716-cecef21b1cd1 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:44.939416 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5278e1d4-558c-442f-a716-cecef21b1cd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:44.970359 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278e1d4-558c-442f-a716-cecef21b1cd1 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:44.995026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b18b0-a216-46ba-9879-994cc74e286f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:44.995881 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b18b0-a216-46ba-9879-994cc74e286f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:44.996157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b18b0-a216-46ba-9879-994cc74e286f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:44.996423 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b18b0-a216-46ba-9879-994cc74e286f admin admin] PATH_INFO: `/v3/projects/62d72bb5612346a68ed3d6f4a524f2a2` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:45.005313 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b18b0-a216-46ba-9879-994cc74e286f admin admin] RBAC: Authorizing `identity:delete_project(project_id=62d72bb5612346a68ed3d6f4a524f2a2)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:45.007297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b18b0-a216-46ba-9879-994cc74e286f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:45.096672 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-fe0b18b0-a216-46ba-9879-994cc74e286f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62d72bb5612346a68ed3d6f4a524f2a2', 'request_id': 'req-fe0b18b0-a216-46ba-9879-994cc74e286f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:45.109481 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-fe0b18b0-a216-46ba-9879-994cc74e286f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62d72bb5612346a68ed3d6f4a524f2a2', 'request_id': 'req-fe0b18b0-a216-46ba-9879-994cc74e286f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:45.117965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-fe0b18b0-a216-46ba-9879-994cc74e286f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62d72bb5612346a68ed3d6f4a524f2a2', 'request_id': 'req-fe0b18b0-a216-46ba-9879-994cc74e286f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:45.141124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-fe0b18b0-a216-46ba-9879-994cc74e286f admin admin] The token cache is being invalidate because project 62d72bb5612346a68ed3d6f4a524f2a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:45.141457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-fe0b18b0-a216-46ba-9879-994cc74e286f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:45.142410 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-fe0b18b0-a216-46ba-9879-994cc74e286f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:45.144952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1108/2205] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:44 2022] DELETE /identity/v3/projects/62d72bb5612346a68ed3d6f4a524f2a2 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:45.150195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6db71-44dc-4c53-a4cf-efcc6e7ff7db service nova] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:45.172868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6db71-44dc-4c53-a4cf-efcc6e7ff7db service nova] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:45.259901 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a815e-d8c7-4e34-9cc7-0269315260e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:45.261358 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a815e-d8c7-4e34-9cc7-0269315260e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:45.261636 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a815e-d8c7-4e34-9cc7-0269315260e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:45.261919 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a815e-d8c7-4e34-9cc7-0269315260e6 admin admin] PATH_INFO: `/v3/users/b5fa58b1e92f47b5b22eb9fecd4f37cc` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:45.264386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a815e-d8c7-4e34-9cc7-0269315260e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5fa58b1e92f47b5b22eb9fecd4f37cc)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:45.268157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a815e-d8c7-4e34-9cc7-0269315260e6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:45.334730 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cd5a815e-d8c7-4e34-9cc7-0269315260e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5fa58b1e92f47b5b22eb9fecd4f37cc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:45.343380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cd5a815e-d8c7-4e34-9cc7-0269315260e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5fa58b1e92f47b5b22eb9fecd4f37cc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:45.344355 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cd5a815e-d8c7-4e34-9cc7-0269315260e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5fa58b1e92f47b5b22eb9fecd4f37cc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:45.355941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cd5a815e-d8c7-4e34-9cc7-0269315260e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5fa58b1e92f47b5b22eb9fecd4f37cc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:45.357734 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cd5a815e-d8c7-4e34-9cc7-0269315260e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5fa58b1e92f47b5b22eb9fecd4f37cc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:45.389723 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1098/2206] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:45 2022] DELETE /identity/v3/users/b5fa58b1e92f47b5b22eb9fecd4f37cc => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:46.235176 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de2572-d7e4-46f3-a13d-e944203ed88b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:46.235724 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de2572-d7e4-46f3-a13d-e944203ed88b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:46.235724 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de2572-d7e4-46f3-a13d-e944203ed88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:46.257790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e6de2572-d7e4-46f3-a13d-e944203ed88b None None] MFA Rules not processed for user `c950754974d748519118d974492f15ce`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:46.260518 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e6de2572-d7e4-46f3-a13d-e944203ed88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:46.333663 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1109/2207] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:46.507517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a815e-d8c7-4e34-9cc7-0269315260e6 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:46.509210 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-cd5a815e-d8c7-4e34-9cc7-0269315260e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:46.708513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a815e-d8c7-4e34-9cc7-0269315260e6 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:46.734755 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf8bd8-b7f6-46e3-a72a-ca565f9b6112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:46.736173 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf8bd8-b7f6-46e3-a72a-ca565f9b6112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:46.737569 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf8bd8-b7f6-46e3-a72a-ca565f9b6112 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:46.738017 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf8bd8-b7f6-46e3-a72a-ca565f9b6112 admin admin] PATH_INFO: `/v3/projects/465639098487433bae134893633f98e3` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:46.742661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf8bd8-b7f6-46e3-a72a-ca565f9b6112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=465639098487433bae134893633f98e3)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:46.747070 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf8bd8-b7f6-46e3-a72a-ca565f9b6112 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:46.862616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-27bf8bd8-b7f6-46e3-a72a-ca565f9b6112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '465639098487433bae134893633f98e3', 'request_id': 'req-27bf8bd8-b7f6-46e3-a72a-ca565f9b6112'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:46.870833 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-27bf8bd8-b7f6-46e3-a72a-ca565f9b6112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '465639098487433bae134893633f98e3', 'request_id': 'req-27bf8bd8-b7f6-46e3-a72a-ca565f9b6112'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:46.877595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-27bf8bd8-b7f6-46e3-a72a-ca565f9b6112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '465639098487433bae134893633f98e3', 'request_id': 'req-27bf8bd8-b7f6-46e3-a72a-ca565f9b6112'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:46.903840 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-27bf8bd8-b7f6-46e3-a72a-ca565f9b6112 admin admin] The token cache is being invalidate because project 465639098487433bae134893633f98e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:46.904402 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-27bf8bd8-b7f6-46e3-a72a-ca565f9b6112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:46.905335 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-27bf8bd8-b7f6-46e3-a72a-ca565f9b6112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:46.907399 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1099/2208] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:46 2022] DELETE /identity/v3/projects/465639098487433bae134893633f98e3 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:46.913189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c83a4-e3db-46db-a3bb-edc28e36603c None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:46.913626 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c83a4-e3db-46db-a3bb-edc28e36603c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:46.914167 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c83a4-e3db-46db-a3bb-edc28e36603c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:46.939141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-1d6c83a4-e3db-46db-a3bb-edc28e36603c None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:46.940755 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1d6c83a4-e3db-46db-a3bb-edc28e36603c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:47.020757 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1110/2209] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:47.030167 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297367c-2711-42d2-bc77-60cce14e2f4a None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:47.030838 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297367c-2711-42d2-bc77-60cce14e2f4a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:47.030838 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297367c-2711-42d2-bc77-60cce14e2f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:47.050034 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-7297367c-2711-42d2-bc77-60cce14e2f4a None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:47.051279 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7297367c-2711-42d2-bc77-60cce14e2f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:47.062152 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1100/2210] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:47.066902 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c83a4-e3db-46db-a3bb-edc28e36603c None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:47.079709 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c83a4-e3db-46db-a3bb-edc28e36603c None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:47.087469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63a778-ebd8-44d2-ac43-1cca0f535017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:47.088557 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63a778-ebd8-44d2-ac43-1cca0f535017 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:47.089073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63a778-ebd8-44d2-ac43-1cca0f535017 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:47.090173 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63a778-ebd8-44d2-ac43-1cca0f535017 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:47.091128 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d63a778-ebd8-44d2-ac43-1cca0f535017 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:03:47.091895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d63a778-ebd8-44d2-ac43-1cca0f535017 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:47.093774 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d63a778-ebd8-44d2-ac43-1cca0f535017 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:47.108962 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1111/2211] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:03:47.113170 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297367c-2711-42d2-bc77-60cce14e2f4a None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:47.126142 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297367c-2711-42d2-bc77-60cce14e2f4a None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:47.140401 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204c782-d150-41a1-bba7-161c63415e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:47.141080 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204c782-d150-41a1-bba7-161c63415e70 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:47.141362 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204c782-d150-41a1-bba7-161c63415e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:47.141638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204c782-d150-41a1-bba7-161c63415e70 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:47.142379 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1204c782-d150-41a1-bba7-161c63415e70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:47.144377 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1204c782-d150-41a1-bba7-161c63415e70 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:47.191852 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1101/2212] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:47 2022] POST /identity/v3/projects => generated 381 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:03:47.193674 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63a778-ebd8-44d2-ac43-1cca0f535017 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:47.216556 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63a778-ebd8-44d2-ac43-1cca0f535017 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:47.338409 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5449fed-07fb-4a26-a5a6-5f1bb2d60a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:47.339094 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5449fed-07fb-4a26-a5a6-5f1bb2d60a5f admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:47.340184 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5449fed-07fb-4a26-a5a6-5f1bb2d60a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:47.340621 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5449fed-07fb-4a26-a5a6-5f1bb2d60a5f admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:47.341660 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5449fed-07fb-4a26-a5a6-5f1bb2d60a5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:47.344515 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5449fed-07fb-4a26-a5a6-5f1bb2d60a5f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:47.401274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1112/2213] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:47 2022] POST /identity/v3/users => generated 291 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:03:47.402480 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204c782-d150-41a1-bba7-161c63415e70 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:47.416725 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204c782-d150-41a1-bba7-161c63415e70 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:47.455469 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f857a-5443-4d26-be7f-01fe8ececd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:47.457375 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f857a-5443-4d26-be7f-01fe8ececd10 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:47.459866 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f857a-5443-4d26-be7f-01fe8ececd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:47.460371 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f857a-5443-4d26-be7f-01fe8ececd10 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:47.461233 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f857a-5443-4d26-be7f-01fe8ececd10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:47.470205 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f857a-5443-4d26-be7f-01fe8ececd10 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:47.513158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1102/2214] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:47.516068 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5449fed-07fb-4a26-a5a6-5f1bb2d60a5f admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:47.528614 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5449fed-07fb-4a26-a5a6-5f1bb2d60a5f admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:47.540614 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598564cf-14ca-4e40-a3b4-4fd988bb5399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:47.541436 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598564cf-14ca-4e40-a3b4-4fd988bb5399 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:47.542106 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598564cf-14ca-4e40-a3b4-4fd988bb5399 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:47.544831 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598564cf-14ca-4e40-a3b4-4fd988bb5399 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:47.545678 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598564cf-14ca-4e40-a3b4-4fd988bb5399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:47.550741 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598564cf-14ca-4e40-a3b4-4fd988bb5399 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:47.569009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1113/2215] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:47.570306 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f857a-5443-4d26-be7f-01fe8ececd10 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:47.587431 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f857a-5443-4d26-be7f-01fe8ececd10 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:47.595006 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181fe70-8ca8-4c34-bc3b-4809df8d3cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:47.596199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181fe70-8ca8-4c34-bc3b-4809df8d3cc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:47.597102 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181fe70-8ca8-4c34-bc3b-4809df8d3cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:47.597952 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181fe70-8ca8-4c34-bc3b-4809df8d3cc8 admin admin] PATH_INFO: `/v3/projects/e55dba1fca8046dfbfcc50987fdcabd0/users/51436cb38b5147c894424e379de4ae84/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:47.627754 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f181fe70-8ca8-4c34-bc3b-4809df8d3cc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e55dba1fca8046dfbfcc50987fdcabd0, user_id=51436cb38b5147c894424e379de4ae84, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:47.632011 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f181fe70-8ca8-4c34-bc3b-4809df8d3cc8 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:47.658281 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1103/2216] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:47 2022] PUT /identity/v3/projects/e55dba1fca8046dfbfcc50987fdcabd0/users/51436cb38b5147c894424e379de4ae84/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:47.668341 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e9f17-3084-497d-be5c-8a5d0f3099b6 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:47.668850 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e9f17-3084-497d-be5c-8a5d0f3099b6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:47.669298 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e9f17-3084-497d-be5c-8a5d0f3099b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:47.710154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e40e9f17-3084-497d-be5c-8a5d0f3099b6 None None] MFA Rules not processed for user `51436cb38b5147c894424e379de4ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:47.711367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e40e9f17-3084-497d-be5c-8a5d0f3099b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:47.817059 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1114/2217] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:47.819971 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181fe70-8ca8-4c34-bc3b-4809df8d3cc8 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:47.939157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181fe70-8ca8-4c34-bc3b-4809df8d3cc8 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:47.944547 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b0586-8be1-43d1-8837-eac8f7090e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:47.945126 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b0586-8be1-43d1-8837-eac8f7090e29 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:47.945403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b0586-8be1-43d1-8837-eac8f7090e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:47.945761 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b0586-8be1-43d1-8837-eac8f7090e29 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:47.946628 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b0586-8be1-43d1-8837-eac8f7090e29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:47.948532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b0586-8be1-43d1-8837-eac8f7090e29 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:47.987889 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e9f17-3084-497d-be5c-8a5d0f3099b6 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:47.995258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1104/2218] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:47 2022] POST /identity/v3/projects => generated 381 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:47.999641 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e9f17-3084-497d-be5c-8a5d0f3099b6 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:48.064828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dec5c5-4915-4902-b9e6-5da174b330b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:48.065425 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dec5c5-4915-4902-b9e6-5da174b330b8 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:48.065705 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dec5c5-4915-4902-b9e6-5da174b330b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:48.065988 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dec5c5-4915-4902-b9e6-5da174b330b8 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:48.066833 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dec5c5-4915-4902-b9e6-5da174b330b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:48.068609 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dec5c5-4915-4902-b9e6-5da174b330b8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:48.089004 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1115/2219] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:47 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:48.093744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b0586-8be1-43d1-8837-eac8f7090e29 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:48.105898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b0586-8be1-43d1-8837-eac8f7090e29 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:48.111032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221eff1-b779-4cfc-ba37-6c79f327bb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:48.111696 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221eff1-b779-4cfc-ba37-6c79f327bb25 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:48.111999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221eff1-b779-4cfc-ba37-6c79f327bb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:48.112271 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221eff1-b779-4cfc-ba37-6c79f327bb25 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:48.113042 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7221eff1-b779-4cfc-ba37-6c79f327bb25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:48.114754 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7221eff1-b779-4cfc-ba37-6c79f327bb25 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:48.130733 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1105/2220] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:48.131679 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dec5c5-4915-4902-b9e6-5da174b330b8 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:48.142834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dec5c5-4915-4902-b9e6-5da174b330b8 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:48.157778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c436f-f630-4e19-81ae-507c8141c891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:48.159877 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c436f-f630-4e19-81ae-507c8141c891 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:48.159877 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c436f-f630-4e19-81ae-507c8141c891 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:48.159877 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c436f-f630-4e19-81ae-507c8141c891 admin admin] PATH_INFO: `/v3/projects/e5690f965b184141aa74bcabe11ec30a/users/6b208407035f495ab9426c3463749298/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:48.182695 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c436f-f630-4e19-81ae-507c8141c891 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5690f965b184141aa74bcabe11ec30a, user_id=6b208407035f495ab9426c3463749298, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:48.188503 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c436f-f630-4e19-81ae-507c8141c891 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:48.211298 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1116/2221] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:48 2022] PUT /identity/v3/projects/e5690f965b184141aa74bcabe11ec30a/users/6b208407035f495ab9426c3463749298/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:48.214933 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221eff1-b779-4cfc-ba37-6c79f327bb25 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:48.227126 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221eff1-b779-4cfc-ba37-6c79f327bb25 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:48.318003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a9292-084c-43b0-b91a-2cb980348f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:48.318722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a9292-084c-43b0-b91a-2cb980348f0c admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:48.319003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a9292-084c-43b0-b91a-2cb980348f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:48.319283 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a9292-084c-43b0-b91a-2cb980348f0c admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:48.320042 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a9292-084c-43b0-b91a-2cb980348f0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:48.321731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a9292-084c-43b0-b91a-2cb980348f0c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:48.349094 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1106/2222] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:48.349746 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c436f-f630-4e19-81ae-507c8141c891 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:48.360755 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c436f-f630-4e19-81ae-507c8141c891 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:48.367201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750c366-f235-43d0-b0f0-8def90b72ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:48.368375 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750c366-f235-43d0-b0f0-8def90b72ff6 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:48.368873 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750c366-f235-43d0-b0f0-8def90b72ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:48.369354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750c366-f235-43d0-b0f0-8def90b72ff6 admin admin] PATH_INFO: `/v3/projects/e5690f965b184141aa74bcabe11ec30a/users/6b208407035f495ab9426c3463749298/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:48.371695 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750c366-f235-43d0-b0f0-8def90b72ff6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5690f965b184141aa74bcabe11ec30a, user_id=6b208407035f495ab9426c3463749298, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:48.375624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750c366-f235-43d0-b0f0-8def90b72ff6 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:48.387699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1117/2223] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:48 2022] PUT /identity/v3/projects/e5690f965b184141aa74bcabe11ec30a/users/6b208407035f495ab9426c3463749298/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:03:48.400256 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be4694-b565-4849-8dd7-c5a4e6f88665 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:48.400677 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be4694-b565-4849-8dd7-c5a4e6f88665 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:48.401060 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be4694-b565-4849-8dd7-c5a4e6f88665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:48.464042 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d6be4694-b565-4849-8dd7-c5a4e6f88665 None None] MFA Rules not processed for user `6b208407035f495ab9426c3463749298`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:48.465841 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d6be4694-b565-4849-8dd7-c5a4e6f88665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:48.556545 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1107/2224] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:03:48.569211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750c366-f235-43d0-b0f0-8def90b72ff6 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:48.570946 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7750c366-f235-43d0-b0f0-8def90b72ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:48.651458 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750c366-f235-43d0-b0f0-8def90b72ff6 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:48.655585 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e00d7-b16e-4c02-9203-22ebb0cc434c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:48.656353 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e00d7-b16e-4c02-9203-22ebb0cc434c service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:48.656899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e00d7-b16e-4c02-9203-22ebb0cc434c service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:48.657241 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e00d7-b16e-4c02-9203-22ebb0cc434c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:48.670458 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e00d7-b16e-4c02-9203-22ebb0cc434c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:48.672758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e00d7-b16e-4c02-9203-22ebb0cc434c service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:48.683794 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1118/2225] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:49.097356 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be4694-b565-4849-8dd7-c5a4e6f88665 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:49.155294 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be4694-b565-4849-8dd7-c5a4e6f88665 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:49.159685 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13388d29-0e93-4801-a8e7-bca6a0c5f53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:49.160523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13388d29-0e93-4801-a8e7-bca6a0c5f53d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:49.160797 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13388d29-0e93-4801-a8e7-bca6a0c5f53d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:49.161063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13388d29-0e93-4801-a8e7-bca6a0c5f53d admin admin] PATH_INFO: `/v3/users/51436cb38b5147c894424e379de4ae84` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:49.162243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13388d29-0e93-4801-a8e7-bca6a0c5f53d admin admin] RBAC: Authorizing `identity:delete_user(user_id=51436cb38b5147c894424e379de4ae84)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:49.164127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13388d29-0e93-4801-a8e7-bca6a0c5f53d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:49.208260 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-13388d29-0e93-4801-a8e7-bca6a0c5f53d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51436cb38b5147c894424e379de4ae84'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:49.214724 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-13388d29-0e93-4801-a8e7-bca6a0c5f53d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51436cb38b5147c894424e379de4ae84'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:49.215420 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-13388d29-0e93-4801-a8e7-bca6a0c5f53d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51436cb38b5147c894424e379de4ae84'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:49.223095 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-13388d29-0e93-4801-a8e7-bca6a0c5f53d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51436cb38b5147c894424e379de4ae84'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:49.223818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-13388d29-0e93-4801-a8e7-bca6a0c5f53d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51436cb38b5147c894424e379de4ae84'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:49.241799 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1108/2226] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:49 2022] DELETE /identity/v3/users/51436cb38b5147c894424e379de4ae84 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:49.246254 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcec51-026f-4665-a7a0-ab7d5b8483c6 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:49.246637 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcec51-026f-4665-a7a0-ab7d5b8483c6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:49.246961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcec51-026f-4665-a7a0-ab7d5b8483c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:49.266812 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-7bfcec51-026f-4665-a7a0-ab7d5b8483c6 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:49.268019 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7bfcec51-026f-4665-a7a0-ab7d5b8483c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:49.328621 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1119/2227] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:49.338351 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13388d29-0e93-4801-a8e7-bca6a0c5f53d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:49.339775 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-13388d29-0e93-4801-a8e7-bca6a0c5f53d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:49.421445 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13388d29-0e93-4801-a8e7-bca6a0c5f53d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:49.429991 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2f91e-8e6d-4688-845c-cbbaa64397bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:49.430674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2f91e-8e6d-4688-845c-cbbaa64397bf service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:49.430983 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2f91e-8e6d-4688-845c-cbbaa64397bf service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:49.431291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2f91e-8e6d-4688-845c-cbbaa64397bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:49.443357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad2f91e-8e6d-4688-845c-cbbaa64397bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:49.444789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad2f91e-8e6d-4688-845c-cbbaa64397bf service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:49.455425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1109/2228] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:49.750261 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfcec51-026f-4665-a7a0-ab7d5b8483c6 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:49.752442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7bfcec51-026f-4665-a7a0-ab7d5b8483c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:49.764379 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfcec51-026f-4665-a7a0-ab7d5b8483c6 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:49.768937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b16419-8e5d-4f37-8fe4-9d8a9747077c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:49.769792 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b16419-8e5d-4f37-8fe4-9d8a9747077c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:49.770275 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b16419-8e5d-4f37-8fe4-9d8a9747077c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:49.770554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b16419-8e5d-4f37-8fe4-9d8a9747077c admin admin] PATH_INFO: `/v3/projects/e55dba1fca8046dfbfcc50987fdcabd0` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:49.771733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b16419-8e5d-4f37-8fe4-9d8a9747077c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e55dba1fca8046dfbfcc50987fdcabd0)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:49.773803 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b16419-8e5d-4f37-8fe4-9d8a9747077c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:49.844543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-53b16419-8e5d-4f37-8fe4-9d8a9747077c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e55dba1fca8046dfbfcc50987fdcabd0', 'request_id': 'req-53b16419-8e5d-4f37-8fe4-9d8a9747077c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:49.852179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-53b16419-8e5d-4f37-8fe4-9d8a9747077c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e55dba1fca8046dfbfcc50987fdcabd0', 'request_id': 'req-53b16419-8e5d-4f37-8fe4-9d8a9747077c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:49.860028 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-53b16419-8e5d-4f37-8fe4-9d8a9747077c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e55dba1fca8046dfbfcc50987fdcabd0', 'request_id': 'req-53b16419-8e5d-4f37-8fe4-9d8a9747077c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:49.876304 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-53b16419-8e5d-4f37-8fe4-9d8a9747077c admin admin] The token cache is being invalidate because project e55dba1fca8046dfbfcc50987fdcabd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:49.876594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-53b16419-8e5d-4f37-8fe4-9d8a9747077c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:49.877244 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-53b16419-8e5d-4f37-8fe4-9d8a9747077c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:49.878815 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1120/2229] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:49 2022] DELETE /identity/v3/projects/e55dba1fca8046dfbfcc50987fdcabd0 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:49.884660 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2f91e-8e6d-4688-845c-cbbaa64397bf service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:49.899917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2f91e-8e6d-4688-845c-cbbaa64397bf service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:49.963848 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3041e3e-81d9-4775-a001-419c0e68d0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:49.965514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3041e3e-81d9-4775-a001-419c0e68d0cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:49.965871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3041e3e-81d9-4775-a001-419c0e68d0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:49.966705 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3041e3e-81d9-4775-a001-419c0e68d0cf admin admin] PATH_INFO: `/v3/users/6b208407035f495ab9426c3463749298` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:49.968667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3041e3e-81d9-4775-a001-419c0e68d0cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b208407035f495ab9426c3463749298)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:49.970874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3041e3e-81d9-4775-a001-419c0e68d0cf admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:50.021216 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a3041e3e-81d9-4775-a001-419c0e68d0cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b208407035f495ab9426c3463749298'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:50.027860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a3041e3e-81d9-4775-a001-419c0e68d0cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b208407035f495ab9426c3463749298'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:50.028532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a3041e3e-81d9-4775-a001-419c0e68d0cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b208407035f495ab9426c3463749298'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:50.036296 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a3041e3e-81d9-4775-a001-419c0e68d0cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b208407035f495ab9426c3463749298'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:50.037059 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a3041e3e-81d9-4775-a001-419c0e68d0cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b208407035f495ab9426c3463749298'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:50.053710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1110/2230] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:49 2022] DELETE /identity/v3/users/6b208407035f495ab9426c3463749298 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:50.312046 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b16419-8e5d-4f37-8fe4-9d8a9747077c admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:50.313936 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-53b16419-8e5d-4f37-8fe4-9d8a9747077c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:50.326948 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b16419-8e5d-4f37-8fe4-9d8a9747077c admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:50.382374 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c298fa-18e9-46aa-b314-708df0ef8f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:50.383223 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c298fa-18e9-46aa-b314-708df0ef8f13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:50.383500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c298fa-18e9-46aa-b314-708df0ef8f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:50.383773 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c298fa-18e9-46aa-b314-708df0ef8f13 admin admin] PATH_INFO: `/v3/projects/e5690f965b184141aa74bcabe11ec30a` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:50.384924 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c298fa-18e9-46aa-b314-708df0ef8f13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5690f965b184141aa74bcabe11ec30a)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:50.386854 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c298fa-18e9-46aa-b314-708df0ef8f13 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:50.430584 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-98c298fa-18e9-46aa-b314-708df0ef8f13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5690f965b184141aa74bcabe11ec30a', 'request_id': 'req-98c298fa-18e9-46aa-b314-708df0ef8f13'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:50.435710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-98c298fa-18e9-46aa-b314-708df0ef8f13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5690f965b184141aa74bcabe11ec30a', 'request_id': 'req-98c298fa-18e9-46aa-b314-708df0ef8f13'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:50.442210 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-98c298fa-18e9-46aa-b314-708df0ef8f13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5690f965b184141aa74bcabe11ec30a', 'request_id': 'req-98c298fa-18e9-46aa-b314-708df0ef8f13'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:50.454107 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-98c298fa-18e9-46aa-b314-708df0ef8f13 admin admin] The token cache is being invalidate because project e5690f965b184141aa74bcabe11ec30a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:50.454430 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-98c298fa-18e9-46aa-b314-708df0ef8f13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:50.455150 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-98c298fa-18e9-46aa-b314-708df0ef8f13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:50.456560 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1121/2231] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:50 2022] DELETE /identity/v3/projects/e5690f965b184141aa74bcabe11ec30a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:50.461336 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c0b7b-609b-47d7-b691-cae38ffa9a8e None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:50.461628 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c0b7b-609b-47d7-b691-cae38ffa9a8e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:50.461917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c0b7b-609b-47d7-b691-cae38ffa9a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:50.480157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-6b9c0b7b-609b-47d7-b691-cae38ffa9a8e None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:50.481421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-6b9c0b7b-609b-47d7-b691-cae38ffa9a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:50.538479 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1111/2232] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:50.546774 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bdb2f-41f9-4bdd-8382-517c9db8ff30 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:50.547051 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bdb2f-41f9-4bdd-8382-517c9db8ff30 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:50.547330 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bdb2f-41f9-4bdd-8382-517c9db8ff30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:50.565832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-545bdb2f-41f9-4bdd-8382-517c9db8ff30 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:50.567045 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-545bdb2f-41f9-4bdd-8382-517c9db8ff30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:50.574915 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1122/2233] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:50.579652 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c0b7b-609b-47d7-b691-cae38ffa9a8e None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:50.591079 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c0b7b-609b-47d7-b691-cae38ffa9a8e None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:50.597220 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac79c97-f4aa-4192-bee6-402bd976b6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:50.597887 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac79c97-f4aa-4192-bee6-402bd976b6d7 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:50.598226 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac79c97-f4aa-4192-bee6-402bd976b6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:50.598498 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac79c97-f4aa-4192-bee6-402bd976b6d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:50.599684 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac79c97-f4aa-4192-bee6-402bd976b6d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:03:50.599684 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac79c97-f4aa-4192-bee6-402bd976b6d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:50.601265 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac79c97-f4aa-4192-bee6-402bd976b6d7 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:50.614497 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1112/2234] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:03:50.618796 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545bdb2f-41f9-4bdd-8382-517c9db8ff30 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:50.631711 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545bdb2f-41f9-4bdd-8382-517c9db8ff30 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:50.636845 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5aa19-75f7-4fe5-94bb-409d4dbab5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:50.637226 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5aa19-75f7-4fe5-94bb-409d4dbab5dd admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:50.637599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5aa19-75f7-4fe5-94bb-409d4dbab5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:50.637964 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5aa19-75f7-4fe5-94bb-409d4dbab5dd admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:50.638769 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb5aa19-75f7-4fe5-94bb-409d4dbab5dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:50.640799 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb5aa19-75f7-4fe5-94bb-409d4dbab5dd admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:50.671361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1123/2235] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:50 2022] POST /identity/v3/projects => generated 361 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:50.672545 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac79c97-f4aa-4192-bee6-402bd976b6d7 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:50.682709 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac79c97-f4aa-4192-bee6-402bd976b6d7 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:50.736840 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f272a52-ae81-4764-9d51-2bc8080a0db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:50.737498 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f272a52-ae81-4764-9d51-2bc8080a0db1 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:50.737781 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f272a52-ae81-4764-9d51-2bc8080a0db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:50.738055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f272a52-ae81-4764-9d51-2bc8080a0db1 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:50.738760 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f272a52-ae81-4764-9d51-2bc8080a0db1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:50.740704 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f272a52-ae81-4764-9d51-2bc8080a0db1 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:50.759255 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1113/2236] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:50 2022] POST /identity/v3/users => generated 281 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:50.763302 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5aa19-75f7-4fe5-94bb-409d4dbab5dd admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:50.773606 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5aa19-75f7-4fe5-94bb-409d4dbab5dd admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:50.778336 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9adec6-e181-4087-8ba3-61c96117d9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:50.778990 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9adec6-e181-4087-8ba3-61c96117d9cb admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:50.779272 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9adec6-e181-4087-8ba3-61c96117d9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:50.779547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9adec6-e181-4087-8ba3-61c96117d9cb admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:50.780800 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9adec6-e181-4087-8ba3-61c96117d9cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:50.782406 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9adec6-e181-4087-8ba3-61c96117d9cb admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:50.792546 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1124/2237] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:50.797153 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f272a52-ae81-4764-9d51-2bc8080a0db1 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:50.807083 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f272a52-ae81-4764-9d51-2bc8080a0db1 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:50.811949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12690bb4-14c1-4099-a947-b2ac194ef180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:50.812532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12690bb4-14c1-4099-a947-b2ac194ef180 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:50.812759 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12690bb4-14c1-4099-a947-b2ac194ef180 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:50.813530 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12690bb4-14c1-4099-a947-b2ac194ef180 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:50.814307 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12690bb4-14c1-4099-a947-b2ac194ef180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:50.815898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12690bb4-14c1-4099-a947-b2ac194ef180 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:50.827558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1114/2238] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:50.831704 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9adec6-e181-4087-8ba3-61c96117d9cb admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:50.841354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9adec6-e181-4087-8ba3-61c96117d9cb admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:50.846465 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a1f0d-e2b1-4a65-b785-f257b090e032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:50.847325 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a1f0d-e2b1-4a65-b785-f257b090e032 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:50.847603 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a1f0d-e2b1-4a65-b785-f257b090e032 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:50.847875 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a1f0d-e2b1-4a65-b785-f257b090e032 admin admin] PATH_INFO: `/v3/projects/f4fd66613b3f431b854cfebf3fa1a686/users/1175c2df54b74d159a0b4a2a9e0c354d/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:50.862998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a1f0d-e2b1-4a65-b785-f257b090e032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4fd66613b3f431b854cfebf3fa1a686, user_id=1175c2df54b74d159a0b4a2a9e0c354d, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:50.865637 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a1f0d-e2b1-4a65-b785-f257b090e032 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:50.878153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1125/2239] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:50 2022] PUT /identity/v3/projects/f4fd66613b3f431b854cfebf3fa1a686/users/1175c2df54b74d159a0b4a2a9e0c354d/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:03:50.888122 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544f454-0446-4f9c-afba-15a1d026849c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:50.888570 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544f454-0446-4f9c-afba-15a1d026849c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:50.888658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544f454-0446-4f9c-afba-15a1d026849c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:50.916213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b544f454-0446-4f9c-afba-15a1d026849c None None] MFA Rules not processed for user `1175c2df54b74d159a0b4a2a9e0c354d`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:50.917450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b544f454-0446-4f9c-afba-15a1d026849c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:50.988530 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1115/2240] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:51.033193 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a1f0d-e2b1-4a65-b785-f257b090e032 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:51.035246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-882a1f0d-e2b1-4a65-b785-f257b090e032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:51.097600 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a1f0d-e2b1-4a65-b785-f257b090e032 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:51.101198 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4dcd70-4f28-4180-ae41-82ccbc67d652 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:51.101834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4dcd70-4f28-4180-ae41-82ccbc67d652 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:51.102154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4dcd70-4f28-4180-ae41-82ccbc67d652 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:51.102426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4dcd70-4f28-4180-ae41-82ccbc67d652 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:51.111298 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4dcd70-4f28-4180-ae41-82ccbc67d652 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:51.112721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4dcd70-4f28-4180-ae41-82ccbc67d652 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:51.121245 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1126/2241] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:51.135435 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544f454-0446-4f9c-afba-15a1d026849c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:51.193638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544f454-0446-4f9c-afba-15a1d026849c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:51.198009 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f6522-2476-4ce3-b2f1-bdc4a9827228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:51.198871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f6522-2476-4ce3-b2f1-bdc4a9827228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:51.199144 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f6522-2476-4ce3-b2f1-bdc4a9827228 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:51.199408 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f6522-2476-4ce3-b2f1-bdc4a9827228 admin admin] PATH_INFO: `/v3/users/1175c2df54b74d159a0b4a2a9e0c354d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:51.200559 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f6522-2476-4ce3-b2f1-bdc4a9827228 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1175c2df54b74d159a0b4a2a9e0c354d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:51.202479 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f6522-2476-4ce3-b2f1-bdc4a9827228 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:51.247434 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-9b8f6522-2476-4ce3-b2f1-bdc4a9827228 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1175c2df54b74d159a0b4a2a9e0c354d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:51.254247 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-9b8f6522-2476-4ce3-b2f1-bdc4a9827228 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1175c2df54b74d159a0b4a2a9e0c354d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:51.254903 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-9b8f6522-2476-4ce3-b2f1-bdc4a9827228 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1175c2df54b74d159a0b4a2a9e0c354d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:51.261874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-9b8f6522-2476-4ce3-b2f1-bdc4a9827228 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1175c2df54b74d159a0b4a2a9e0c354d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:51.262578 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-9b8f6522-2476-4ce3-b2f1-bdc4a9827228 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1175c2df54b74d159a0b4a2a9e0c354d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:51.279140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1116/2242] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:51 2022] DELETE /identity/v3/users/1175c2df54b74d159a0b4a2a9e0c354d => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:51.284213 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c826e5a-d1fb-4c14-a3c4-ae00000dbacf None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:51.284505 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c826e5a-d1fb-4c14-a3c4-ae00000dbacf None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:51.284784 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c826e5a-d1fb-4c14-a3c4-ae00000dbacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:51.302356 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-0c826e5a-d1fb-4c14-a3c4-ae00000dbacf None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:51.303541 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0c826e5a-d1fb-4c14-a3c4-ae00000dbacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:51.362059 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1127/2243] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:51.371771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f6522-2476-4ce3-b2f1-bdc4a9827228 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:51.373134 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9b8f6522-2476-4ce3-b2f1-bdc4a9827228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:51.428262 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f6522-2476-4ce3-b2f1-bdc4a9827228 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:51.431903 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc827290-f940-473a-8309-2417d4c77b1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:51.432534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc827290-f940-473a-8309-2417d4c77b1b service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:51.432811 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc827290-f940-473a-8309-2417d4c77b1b service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:51.433082 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc827290-f940-473a-8309-2417d4c77b1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:51.441784 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc827290-f940-473a-8309-2417d4c77b1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:51.443235 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc827290-f940-473a-8309-2417d4c77b1b service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:51.452431 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1117/2244] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:51.969942 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c826e5a-d1fb-4c14-a3c4-ae00000dbacf None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:51.971398 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0c826e5a-d1fb-4c14-a3c4-ae00000dbacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:51.984046 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c826e5a-d1fb-4c14-a3c4-ae00000dbacf None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:51.995198 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4720b6-835e-42e0-9e0e-654be55a944c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:51.996479 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4720b6-835e-42e0-9e0e-654be55a944c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:51.997134 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4720b6-835e-42e0-9e0e-654be55a944c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:51.997571 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4720b6-835e-42e0-9e0e-654be55a944c admin admin] PATH_INFO: `/v3/projects/f4fd66613b3f431b854cfebf3fa1a686` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:51.999499 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4720b6-835e-42e0-9e0e-654be55a944c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4fd66613b3f431b854cfebf3fa1a686)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:52.002063 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4720b6-835e-42e0-9e0e-654be55a944c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:52.057378 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-3b4720b6-835e-42e0-9e0e-654be55a944c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4fd66613b3f431b854cfebf3fa1a686', 'request_id': 'req-3b4720b6-835e-42e0-9e0e-654be55a944c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:52.083795 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-3b4720b6-835e-42e0-9e0e-654be55a944c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4fd66613b3f431b854cfebf3fa1a686', 'request_id': 'req-3b4720b6-835e-42e0-9e0e-654be55a944c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:52.093158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-3b4720b6-835e-42e0-9e0e-654be55a944c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4fd66613b3f431b854cfebf3fa1a686', 'request_id': 'req-3b4720b6-835e-42e0-9e0e-654be55a944c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:52.115400 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-3b4720b6-835e-42e0-9e0e-654be55a944c admin admin] The token cache is being invalidate because project f4fd66613b3f431b854cfebf3fa1a686 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:52.115694 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-3b4720b6-835e-42e0-9e0e-654be55a944c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:52.116658 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-3b4720b6-835e-42e0-9e0e-654be55a944c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:52.118789 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1128/2245] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:51 2022] DELETE /identity/v3/projects/f4fd66613b3f431b854cfebf3fa1a686 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:52.127673 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098affb-a69b-4862-9cbf-d37b167d7626 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:52.128172 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098affb-a69b-4862-9cbf-d37b167d7626 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:52.128293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098affb-a69b-4862-9cbf-d37b167d7626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:52.148457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d098affb-a69b-4862-9cbf-d37b167d7626 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:52.149778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d098affb-a69b-4862-9cbf-d37b167d7626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:52.211511 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1118/2246] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:52.220358 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785e4fc-6578-4516-b4f7-0724a3a8e44f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:52.220585 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785e4fc-6578-4516-b4f7-0724a3a8e44f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:52.220869 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785e4fc-6578-4516-b4f7-0724a3a8e44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:52.243780 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-d785e4fc-6578-4516-b4f7-0724a3a8e44f None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:52.245068 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d785e4fc-6578-4516-b4f7-0724a3a8e44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:52.253891 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1129/2247] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:52.257850 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d098affb-a69b-4862-9cbf-d37b167d7626 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:52.268676 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d098affb-a69b-4862-9cbf-d37b167d7626 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:52.273215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6eae7-d4b7-4f5a-828c-b9133c88c3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:52.273872 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6eae7-d4b7-4f5a-828c-b9133c88c3c0 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:52.274166 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6eae7-d4b7-4f5a-828c-b9133c88c3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:52.274437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6eae7-d4b7-4f5a-828c-b9133c88c3c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:52.275149 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6eae7-d4b7-4f5a-828c-b9133c88c3c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:03:52.275567 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6eae7-d4b7-4f5a-828c-b9133c88c3c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:52.277151 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6eae7-d4b7-4f5a-828c-b9133c88c3c0 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:52.289207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1119/2248] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:03:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:03:52.293349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785e4fc-6578-4516-b4f7-0724a3a8e44f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:52.302757 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785e4fc-6578-4516-b4f7-0724a3a8e44f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:52.307405 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53314068-61fa-47c4-aa5f-4270e7219d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:52.308056 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53314068-61fa-47c4-aa5f-4270e7219d6d admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:52.308334 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53314068-61fa-47c4-aa5f-4270e7219d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:52.308608 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53314068-61fa-47c4-aa5f-4270e7219d6d admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:52.309335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53314068-61fa-47c4-aa5f-4270e7219d6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:52.311262 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53314068-61fa-47c4-aa5f-4270e7219d6d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:52.338441 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1130/2249] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:52 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:03:52.342797 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6eae7-d4b7-4f5a-828c-b9133c88c3c0 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:52.353848 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6eae7-d4b7-4f5a-828c-b9133c88c3c0 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:52.411371 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e05548-7330-4371-bada-43c52218627d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:52.412042 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e05548-7330-4371-bada-43c52218627d admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:52.412323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e05548-7330-4371-bada-43c52218627d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:52.412600 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e05548-7330-4371-bada-43c52218627d admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:52.413314 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e05548-7330-4371-bada-43c52218627d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:52.415280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e05548-7330-4371-bada-43c52218627d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:52.435982 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1120/2250] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:52 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:52.440546 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53314068-61fa-47c4-aa5f-4270e7219d6d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:52.450922 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53314068-61fa-47c4-aa5f-4270e7219d6d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:52.456153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48188fcd-decc-4152-93fb-ef67be2c2305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:52.456814 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48188fcd-decc-4152-93fb-ef67be2c2305 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:52.457113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48188fcd-decc-4152-93fb-ef67be2c2305 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:52.457392 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48188fcd-decc-4152-93fb-ef67be2c2305 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:52.458135 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48188fcd-decc-4152-93fb-ef67be2c2305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:52.459787 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48188fcd-decc-4152-93fb-ef67be2c2305 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:52.470683 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1131/2251] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:52.474501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e05548-7330-4371-bada-43c52218627d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:52.484683 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e05548-7330-4371-bada-43c52218627d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:52.489360 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef4edb-5337-48be-874f-a3ded4839fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:52.490021 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef4edb-5337-48be-874f-a3ded4839fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:52.490334 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef4edb-5337-48be-874f-a3ded4839fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:52.490613 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef4edb-5337-48be-874f-a3ded4839fe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:52.491376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ef4edb-5337-48be-874f-a3ded4839fe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:52.492984 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ef4edb-5337-48be-874f-a3ded4839fe9 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:52.507390 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1121/2252] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:52.511743 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48188fcd-decc-4152-93fb-ef67be2c2305 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:52.522371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48188fcd-decc-4152-93fb-ef67be2c2305 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:52.527893 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2dae24-ef5e-437b-bf64-cd9603448a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:52.528936 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dae24-ef5e-437b-bf64-cd9603448a15 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:52.529413 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dae24-ef5e-437b-bf64-cd9603448a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:52.529864 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dae24-ef5e-437b-bf64-cd9603448a15 admin admin] PATH_INFO: `/v3/projects/a10894d1a0b54c95afbb01ed1464b3d2/users/42181a322abe41abb1554c1eae61801b/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:52.546914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2dae24-ef5e-437b-bf64-cd9603448a15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a10894d1a0b54c95afbb01ed1464b3d2, user_id=42181a322abe41abb1554c1eae61801b, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:52.549742 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2dae24-ef5e-437b-bf64-cd9603448a15 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:52.568182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1132/2253] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:52 2022] PUT /identity/v3/projects/a10894d1a0b54c95afbb01ed1464b3d2/users/42181a322abe41abb1554c1eae61801b/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:03:52.579613 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42f838-8e61-4b16-abab-6c36cf0c6622 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:52.579974 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42f838-8e61-4b16-abab-6c36cf0c6622 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:52.580487 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42f838-8e61-4b16-abab-6c36cf0c6622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:52.620305 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-7c42f838-8e61-4b16-abab-6c36cf0c6622 None None] MFA Rules not processed for user `42181a322abe41abb1554c1eae61801b`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:52.626196 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7c42f838-8e61-4b16-abab-6c36cf0c6622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:52.695403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1122/2254] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:52.700049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2dae24-ef5e-437b-bf64-cd9603448a15 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:52.766541 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2dae24-ef5e-437b-bf64-cd9603448a15 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:52.771573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cf79f-6e53-414c-ae7e-4b111275afc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:52.772322 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cf79f-6e53-414c-ae7e-4b111275afc0 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:52.772675 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cf79f-6e53-414c-ae7e-4b111275afc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:52.773025 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cf79f-6e53-414c-ae7e-4b111275afc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:52.773797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800cf79f-6e53-414c-ae7e-4b111275afc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:52.775832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800cf79f-6e53-414c-ae7e-4b111275afc0 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:52.804213 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1133/2255] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:52 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:52.808301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42f838-8e61-4b16-abab-6c36cf0c6622 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:52.819648 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42f838-8e61-4b16-abab-6c36cf0c6622 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:52.879768 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11b937-847f-41a8-a2e8-09d9e50f0be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:52.880373 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11b937-847f-41a8-a2e8-09d9e50f0be5 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:52.880649 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11b937-847f-41a8-a2e8-09d9e50f0be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:52.880930 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11b937-847f-41a8-a2e8-09d9e50f0be5 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:52.881639 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11b937-847f-41a8-a2e8-09d9e50f0be5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:52.883668 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11b937-847f-41a8-a2e8-09d9e50f0be5 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:52.909180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1123/2256] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:52 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:52.913048 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cf79f-6e53-414c-ae7e-4b111275afc0 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:52.924304 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cf79f-6e53-414c-ae7e-4b111275afc0 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:52.929396 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c5bf1-773b-40bc-a116-ef8c506ffeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:52.930060 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c5bf1-773b-40bc-a116-ef8c506ffeb2 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:52.930375 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c5bf1-773b-40bc-a116-ef8c506ffeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:52.930651 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c5bf1-773b-40bc-a116-ef8c506ffeb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:52.931376 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c5bf1-773b-40bc-a116-ef8c506ffeb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:52.933018 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c5bf1-773b-40bc-a116-ef8c506ffeb2 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:52.946820 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1134/2257] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:52.948169 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11b937-847f-41a8-a2e8-09d9e50f0be5 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:52.958154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11b937-847f-41a8-a2e8-09d9e50f0be5 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:52.963598 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ad580-2fd7-4be4-b7f8-0e20ca86a3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:52.964529 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ad580-2fd7-4be4-b7f8-0e20ca86a3b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:52.964807 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ad580-2fd7-4be4-b7f8-0e20ca86a3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:52.965080 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ad580-2fd7-4be4-b7f8-0e20ca86a3b2 admin admin] PATH_INFO: `/v3/projects/8ebe2057ac1b404780bcaa096d6ffed9/users/7fb15145b7a54872b626927ba466026d/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:52.982261 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ad580-2fd7-4be4-b7f8-0e20ca86a3b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ebe2057ac1b404780bcaa096d6ffed9, user_id=7fb15145b7a54872b626927ba466026d, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:52.985598 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ad580-2fd7-4be4-b7f8-0e20ca86a3b2 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:53.002671 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1124/2258] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:52 2022] PUT /identity/v3/projects/8ebe2057ac1b404780bcaa096d6ffed9/users/7fb15145b7a54872b626927ba466026d/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:53.004043 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c5bf1-773b-40bc-a116-ef8c506ffeb2 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:53.013901 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c5bf1-773b-40bc-a116-ef8c506ffeb2 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:53.083646 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c891164-2f2a-46f4-add1-f0e5d33e2077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:53.084344 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c891164-2f2a-46f4-add1-f0e5d33e2077 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:53.084776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c891164-2f2a-46f4-add1-f0e5d33e2077 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:53.085384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c891164-2f2a-46f4-add1-f0e5d33e2077 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:53.087560 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c891164-2f2a-46f4-add1-f0e5d33e2077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:53.089861 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c891164-2f2a-46f4-add1-f0e5d33e2077 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:53.105724 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1135/2259] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:53.107722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ad580-2fd7-4be4-b7f8-0e20ca86a3b2 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:53.125390 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ad580-2fd7-4be4-b7f8-0e20ca86a3b2 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:53.134977 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c04bf-4959-4919-9d28-1f4cb9547032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:53.135870 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c04bf-4959-4919-9d28-1f4cb9547032 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:53.136149 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c04bf-4959-4919-9d28-1f4cb9547032 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:53.136418 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c04bf-4959-4919-9d28-1f4cb9547032 admin admin] PATH_INFO: `/v3/projects/8ebe2057ac1b404780bcaa096d6ffed9/users/7fb15145b7a54872b626927ba466026d/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:53.138769 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c04bf-4959-4919-9d28-1f4cb9547032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ebe2057ac1b404780bcaa096d6ffed9, user_id=7fb15145b7a54872b626927ba466026d, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:53.149944 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c04bf-4959-4919-9d28-1f4cb9547032 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:53.177213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1125/2260] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:53 2022] PUT /identity/v3/projects/8ebe2057ac1b404780bcaa096d6ffed9/users/7fb15145b7a54872b626927ba466026d/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:53.187329 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727fb4d-7b7d-465a-a3e9-255e1699977c None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:53.187657 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727fb4d-7b7d-465a-a3e9-255e1699977c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:53.187940 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727fb4d-7b7d-465a-a3e9-255e1699977c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:53.231614 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-a727fb4d-7b7d-465a-a3e9-255e1699977c None None] MFA Rules not processed for user `7fb15145b7a54872b626927ba466026d`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:53.232983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a727fb4d-7b7d-465a-a3e9-255e1699977c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:53.322153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1136/2261] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:53 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:53.335248 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c04bf-4959-4919-9d28-1f4cb9547032 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:53.337302 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-205c04bf-4959-4919-9d28-1f4cb9547032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:53.398940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c04bf-4959-4919-9d28-1f4cb9547032 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:53.410439 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4075a-e444-4724-836a-16d65c891406 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:53.411122 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4075a-e444-4724-836a-16d65c891406 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:53.411955 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4075a-e444-4724-836a-16d65c891406 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:53.412287 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4075a-e444-4724-836a-16d65c891406 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:53.486369 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4075a-e444-4724-836a-16d65c891406 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:53.487847 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4075a-e444-4724-836a-16d65c891406 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:53.498409 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1126/2262] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:56.071960 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727fb4d-7b7d-465a-a3e9-255e1699977c None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:56.132229 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727fb4d-7b7d-465a-a3e9-255e1699977c None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:56.136648 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d23e21-adee-4fa8-b41e-7234a2fa9912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:56.137494 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d23e21-adee-4fa8-b41e-7234a2fa9912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:56.137780 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d23e21-adee-4fa8-b41e-7234a2fa9912 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:56.138050 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d23e21-adee-4fa8-b41e-7234a2fa9912 admin admin] PATH_INFO: `/v3/users/42181a322abe41abb1554c1eae61801b` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:56.139240 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d23e21-adee-4fa8-b41e-7234a2fa9912 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42181a322abe41abb1554c1eae61801b)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:56.141724 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d23e21-adee-4fa8-b41e-7234a2fa9912 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:56.186432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-71d23e21-adee-4fa8-b41e-7234a2fa9912 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42181a322abe41abb1554c1eae61801b'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:56.193013 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-71d23e21-adee-4fa8-b41e-7234a2fa9912 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42181a322abe41abb1554c1eae61801b'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:56.200506 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-71d23e21-adee-4fa8-b41e-7234a2fa9912 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42181a322abe41abb1554c1eae61801b'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:56.201276 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-71d23e21-adee-4fa8-b41e-7234a2fa9912 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42181a322abe41abb1554c1eae61801b'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:56.213272 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-71d23e21-adee-4fa8-b41e-7234a2fa9912 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42181a322abe41abb1554c1eae61801b'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:56.219530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1137/2263] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:56 2022] DELETE /identity/v3/users/42181a322abe41abb1554c1eae61801b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:56.224090 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cf015-bfaa-4aa7-8c14-676c04d19df1 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:56.224588 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cf015-bfaa-4aa7-8c14-676c04d19df1 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:56.224682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cf015-bfaa-4aa7-8c14-676c04d19df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:56.241459 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-009cf015-bfaa-4aa7-8c14-676c04d19df1 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:56.242665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-009cf015-bfaa-4aa7-8c14-676c04d19df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:56.297904 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1127/2264] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:56.307662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d23e21-adee-4fa8-b41e-7234a2fa9912 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:56.309080 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-71d23e21-adee-4fa8-b41e-7234a2fa9912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:56.364027 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d23e21-adee-4fa8-b41e-7234a2fa9912 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:56.367610 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27421ad0-7dcd-4e13-913f-349de798025c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:56.368243 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27421ad0-7dcd-4e13-913f-349de798025c service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:56.368524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27421ad0-7dcd-4e13-913f-349de798025c service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:56.368799 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27421ad0-7dcd-4e13-913f-349de798025c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:56.377555 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27421ad0-7dcd-4e13-913f-349de798025c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:56.379003 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27421ad0-7dcd-4e13-913f-349de798025c service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:56.388112 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1138/2265] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:03:56.653575 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cf015-bfaa-4aa7-8c14-676c04d19df1 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:56.664748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-009cf015-bfaa-4aa7-8c14-676c04d19df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:56.693804 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cf015-bfaa-4aa7-8c14-676c04d19df1 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:56.703200 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81181a1c-0224-466c-b1a3-ece2b726f160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:56.704087 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81181a1c-0224-466c-b1a3-ece2b726f160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:56.704636 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81181a1c-0224-466c-b1a3-ece2b726f160 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:56.704910 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81181a1c-0224-466c-b1a3-ece2b726f160 admin admin] PATH_INFO: `/v3/projects/a10894d1a0b54c95afbb01ed1464b3d2` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:56.707060 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81181a1c-0224-466c-b1a3-ece2b726f160 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a10894d1a0b54c95afbb01ed1464b3d2)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:56.708998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81181a1c-0224-466c-b1a3-ece2b726f160 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:56.771578 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-81181a1c-0224-466c-b1a3-ece2b726f160 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a10894d1a0b54c95afbb01ed1464b3d2', 'request_id': 'req-81181a1c-0224-466c-b1a3-ece2b726f160'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:56.781082 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-81181a1c-0224-466c-b1a3-ece2b726f160 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a10894d1a0b54c95afbb01ed1464b3d2', 'request_id': 'req-81181a1c-0224-466c-b1a3-ece2b726f160'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:56.788558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-81181a1c-0224-466c-b1a3-ece2b726f160 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a10894d1a0b54c95afbb01ed1464b3d2', 'request_id': 'req-81181a1c-0224-466c-b1a3-ece2b726f160'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:56.813988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-81181a1c-0224-466c-b1a3-ece2b726f160 admin admin] The token cache is being invalidate because project a10894d1a0b54c95afbb01ed1464b3d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:56.813988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-81181a1c-0224-466c-b1a3-ece2b726f160 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:56.815294 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-81181a1c-0224-466c-b1a3-ece2b726f160 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:56.820981 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1128/2266] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:56 2022] DELETE /identity/v3/projects/a10894d1a0b54c95afbb01ed1464b3d2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:56.822382 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27421ad0-7dcd-4e13-913f-349de798025c service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:56.833187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27421ad0-7dcd-4e13-913f-349de798025c service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:56.907276 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2e77c-afa5-428f-88c2-c8ffce38e920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:56.908534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2e77c-afa5-428f-88c2-c8ffce38e920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:56.909119 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2e77c-afa5-428f-88c2-c8ffce38e920 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:56.909350 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2e77c-afa5-428f-88c2-c8ffce38e920 admin admin] PATH_INFO: `/v3/users/7fb15145b7a54872b626927ba466026d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:56.911157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b2e77c-afa5-428f-88c2-c8ffce38e920 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fb15145b7a54872b626927ba466026d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:56.913069 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b2e77c-afa5-428f-88c2-c8ffce38e920 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:56.969444 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-92b2e77c-afa5-428f-88c2-c8ffce38e920 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fb15145b7a54872b626927ba466026d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:56.977786 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-92b2e77c-afa5-428f-88c2-c8ffce38e920 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fb15145b7a54872b626927ba466026d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:56.991595 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-92b2e77c-afa5-428f-88c2-c8ffce38e920 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fb15145b7a54872b626927ba466026d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:56.992328 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-92b2e77c-afa5-428f-88c2-c8ffce38e920 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fb15145b7a54872b626927ba466026d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:57.006068 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-92b2e77c-afa5-428f-88c2-c8ffce38e920 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fb15145b7a54872b626927ba466026d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:57.015009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1139/2267] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:03:56 2022] DELETE /identity/v3/users/7fb15145b7a54872b626927ba466026d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:57.335231 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81181a1c-0224-466c-b1a3-ece2b726f160 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:57.336764 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-81181a1c-0224-466c-b1a3-ece2b726f160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:57.351842 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81181a1c-0224-466c-b1a3-ece2b726f160 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:57.407769 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e0166-dd89-4c4e-a148-c1a7bfbbc177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:57.408635 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e0166-dd89-4c4e-a148-c1a7bfbbc177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:57.408908 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e0166-dd89-4c4e-a148-c1a7bfbbc177 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:57.409176 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e0166-dd89-4c4e-a148-c1a7bfbbc177 admin admin] PATH_INFO: `/v3/projects/8ebe2057ac1b404780bcaa096d6ffed9` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:57.410360 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e0166-dd89-4c4e-a148-c1a7bfbbc177 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ebe2057ac1b404780bcaa096d6ffed9)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:57.412287 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e0166-dd89-4c4e-a148-c1a7bfbbc177 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:57.454733 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ed4e0166-dd89-4c4e-a148-c1a7bfbbc177 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ebe2057ac1b404780bcaa096d6ffed9', 'request_id': 'req-ed4e0166-dd89-4c4e-a148-c1a7bfbbc177'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:57.460423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ed4e0166-dd89-4c4e-a148-c1a7bfbbc177 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ebe2057ac1b404780bcaa096d6ffed9', 'request_id': 'req-ed4e0166-dd89-4c4e-a148-c1a7bfbbc177'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:57.465508 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ed4e0166-dd89-4c4e-a148-c1a7bfbbc177 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ebe2057ac1b404780bcaa096d6ffed9', 'request_id': 'req-ed4e0166-dd89-4c4e-a148-c1a7bfbbc177'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:57.477060 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ed4e0166-dd89-4c4e-a148-c1a7bfbbc177 admin admin] The token cache is being invalidate because project 8ebe2057ac1b404780bcaa096d6ffed9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:03:57.477351 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ed4e0166-dd89-4c4e-a148-c1a7bfbbc177 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:57.477975 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ed4e0166-dd89-4c4e-a148-c1a7bfbbc177 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:03:57.479529 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1129/2268] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:03:57 2022] DELETE /identity/v3/projects/8ebe2057ac1b404780bcaa096d6ffed9 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:57.484658 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6a84e-fd22-49ad-bbec-9bbce4081712 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:57.485128 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6a84e-fd22-49ad-bbec-9bbce4081712 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:57.485215 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6a84e-fd22-49ad-bbec-9bbce4081712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:57.503189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-1eb6a84e-fd22-49ad-bbec-9bbce4081712 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:57.504387 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1eb6a84e-fd22-49ad-bbec-9bbce4081712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:57.560784 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1140/2269] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:57.569960 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb41d8-09fa-4cce-b848-dec386599291 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:57.570275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb41d8-09fa-4cce-b848-dec386599291 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:57.570551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb41d8-09fa-4cce-b848-dec386599291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:57.588614 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-54bb41d8-09fa-4cce-b848-dec386599291 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:57.589792 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-54bb41d8-09fa-4cce-b848-dec386599291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:57.598051 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1130/2270] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:57.602054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6a84e-fd22-49ad-bbec-9bbce4081712 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:57.612360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6a84e-fd22-49ad-bbec-9bbce4081712 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:57.616879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e791784-8ebf-4430-ab65-ff6139685ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:57.617797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e791784-8ebf-4430-ab65-ff6139685ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:57.618077 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e791784-8ebf-4430-ab65-ff6139685ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:57.618384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e791784-8ebf-4430-ab65-ff6139685ac1 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:57.619126 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e791784-8ebf-4430-ab65-ff6139685ac1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:03:57.619432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e791784-8ebf-4430-ab65-ff6139685ac1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:57.621020 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e791784-8ebf-4430-ab65-ff6139685ac1 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:57.632294 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1141/2271] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:03:57.636496 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb41d8-09fa-4cce-b848-dec386599291 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:57.645949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb41d8-09fa-4cce-b848-dec386599291 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:57.650543 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6a2bf-5907-41c3-bf72-78f14f3ab985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:57.651174 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6a2bf-5907-41c3-bf72-78f14f3ab985 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:57.651446 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6a2bf-5907-41c3-bf72-78f14f3ab985 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:57.651712 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6a2bf-5907-41c3-bf72-78f14f3ab985 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:57.652387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa6a2bf-5907-41c3-bf72-78f14f3ab985 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:57.654293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa6a2bf-5907-41c3-bf72-78f14f3ab985 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:57.679225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1131/2272] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:57 2022] POST /identity/v3/projects => generated 385 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:57.683195 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e791784-8ebf-4430-ab65-ff6139685ac1 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:57.693032 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e791784-8ebf-4430-ab65-ff6139685ac1 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:57.747170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097b10b-3fbc-4108-8d38-880eec18278e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:57.747828 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097b10b-3fbc-4108-8d38-880eec18278e admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:57.748107 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097b10b-3fbc-4108-8d38-880eec18278e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:57.748380 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097b10b-3fbc-4108-8d38-880eec18278e admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:57.749057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097b10b-3fbc-4108-8d38-880eec18278e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:57.751017 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097b10b-3fbc-4108-8d38-880eec18278e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:57.770213 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1142/2273] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:57 2022] POST /identity/v3/users => generated 293 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:57.774615 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6a2bf-5907-41c3-bf72-78f14f3ab985 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:57.785274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6a2bf-5907-41c3-bf72-78f14f3ab985 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:57.790083 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45668fe1-3f80-440d-ba42-bcccef03d649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:57.790961 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45668fe1-3f80-440d-ba42-bcccef03d649 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:57.791470 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45668fe1-3f80-440d-ba42-bcccef03d649 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:57.791953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45668fe1-3f80-440d-ba42-bcccef03d649 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:57.793263 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45668fe1-3f80-440d-ba42-bcccef03d649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:57.794943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45668fe1-3f80-440d-ba42-bcccef03d649 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:57.808104 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1132/2274] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:57.813053 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097b10b-3fbc-4108-8d38-880eec18278e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:57.825135 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097b10b-3fbc-4108-8d38-880eec18278e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:57.830372 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9979e5-fa77-451c-a4b0-8566a62bba31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:57.831031 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9979e5-fa77-451c-a4b0-8566a62bba31 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:57.831315 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9979e5-fa77-451c-a4b0-8566a62bba31 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:57.831712 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9979e5-fa77-451c-a4b0-8566a62bba31 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:57.832438 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9979e5-fa77-451c-a4b0-8566a62bba31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:57.834063 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9979e5-fa77-451c-a4b0-8566a62bba31 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:57.844168 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1143/2275] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:57.848789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45668fe1-3f80-440d-ba42-bcccef03d649 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:57.858983 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45668fe1-3f80-440d-ba42-bcccef03d649 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:57.863704 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd13157-be33-4452-83ce-5fa2b6ea8590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:57.864564 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd13157-be33-4452-83ce-5fa2b6ea8590 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:57.864839 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd13157-be33-4452-83ce-5fa2b6ea8590 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:57.865105 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd13157-be33-4452-83ce-5fa2b6ea8590 admin admin] PATH_INFO: `/v3/projects/e046fbe4d4c24b429f85f8eee72423bf/users/695871ce6a9c409c868d52cff37b090f/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:57.879387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd13157-be33-4452-83ce-5fa2b6ea8590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e046fbe4d4c24b429f85f8eee72423bf, user_id=695871ce6a9c409c868d52cff37b090f, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:57.882039 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd13157-be33-4452-83ce-5fa2b6ea8590 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:57.895320 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1133/2276] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:57 2022] PUT /identity/v3/projects/e046fbe4d4c24b429f85f8eee72423bf/users/695871ce6a9c409c868d52cff37b090f/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:57.906305 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363e16a-1fa6-4fc8-b2b0-abb0991b389c None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:57.906609 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363e16a-1fa6-4fc8-b2b0-abb0991b389c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:57.906883 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363e16a-1fa6-4fc8-b2b0-abb0991b389c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:57.933534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-9363e16a-1fa6-4fc8-b2b0-abb0991b389c None None] MFA Rules not processed for user `695871ce6a9c409c868d52cff37b090f`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:57.934712 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9363e16a-1fa6-4fc8-b2b0-abb0991b389c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:57.997660 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1144/2277] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:58.002720 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd13157-be33-4452-83ce-5fa2b6ea8590 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:58.065674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd13157-be33-4452-83ce-5fa2b6ea8590 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:58.070362 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b3752-9525-43ad-8d05-f5a1a48bac6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:58.071029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b3752-9525-43ad-8d05-f5a1a48bac6a admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:58.071687 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b3752-9525-43ad-8d05-f5a1a48bac6a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:58.071687 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b3752-9525-43ad-8d05-f5a1a48bac6a admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:58.072240 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b3752-9525-43ad-8d05-f5a1a48bac6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:58.074154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b3752-9525-43ad-8d05-f5a1a48bac6a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:58.102163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1134/2278] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:58 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:03:58.103371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363e16a-1fa6-4fc8-b2b0-abb0991b389c None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:58.114550 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363e16a-1fa6-4fc8-b2b0-abb0991b389c None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:58.175474 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2eb6f-66fe-4c6e-9131-105ea448fb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:58.176157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2eb6f-66fe-4c6e-9131-105ea448fb62 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:58.176438 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2eb6f-66fe-4c6e-9131-105ea448fb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:58.176716 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2eb6f-66fe-4c6e-9131-105ea448fb62 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:58.177413 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb2eb6f-66fe-4c6e-9131-105ea448fb62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:58.179396 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb2eb6f-66fe-4c6e-9131-105ea448fb62 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:58.209346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1145/2279] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:58 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:58.213769 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b3752-9525-43ad-8d05-f5a1a48bac6a admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:58.224837 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b3752-9525-43ad-8d05-f5a1a48bac6a admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:58.232561 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982ce82-7ad3-4640-8bbc-ed81ef8c90a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:58.233338 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982ce82-7ad3-4640-8bbc-ed81ef8c90a9 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:58.233679 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982ce82-7ad3-4640-8bbc-ed81ef8c90a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:58.234283 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982ce82-7ad3-4640-8bbc-ed81ef8c90a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:58.234865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d982ce82-7ad3-4640-8bbc-ed81ef8c90a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:58.238850 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d982ce82-7ad3-4640-8bbc-ed81ef8c90a9 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:58.251453 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1135/2280] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:58.255479 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2eb6f-66fe-4c6e-9131-105ea448fb62 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:58.268878 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2eb6f-66fe-4c6e-9131-105ea448fb62 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:58.273746 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a8105-8d6a-47bc-95cd-06e5aa7c3a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:58.274460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a8105-8d6a-47bc-95cd-06e5aa7c3a52 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:58.274743 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a8105-8d6a-47bc-95cd-06e5aa7c3a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:58.275016 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a8105-8d6a-47bc-95cd-06e5aa7c3a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:58.275874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a8105-8d6a-47bc-95cd-06e5aa7c3a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:58.277487 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a8105-8d6a-47bc-95cd-06e5aa7c3a52 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:58.287876 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1146/2281] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:58.291944 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982ce82-7ad3-4640-8bbc-ed81ef8c90a9 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:58.302471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982ce82-7ad3-4640-8bbc-ed81ef8c90a9 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:58.307558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede5de5-a685-4654-a05a-287bde31fefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:58.308464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede5de5-a685-4654-a05a-287bde31fefa admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:58.308742 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede5de5-a685-4654-a05a-287bde31fefa admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:58.309016 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede5de5-a685-4654-a05a-287bde31fefa admin admin] PATH_INFO: `/v3/projects/89be2e7ed3654f338f25eeabfea5673b/users/1090776a11594f0da41468f30518a67e/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:58.333472 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cede5de5-a685-4654-a05a-287bde31fefa admin admin] RBAC: Authorizing `identity:create_grant(project_id=89be2e7ed3654f338f25eeabfea5673b, user_id=1090776a11594f0da41468f30518a67e, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:58.336231 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cede5de5-a685-4654-a05a-287bde31fefa admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:58.351015 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1136/2282] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:58 2022] PUT /identity/v3/projects/89be2e7ed3654f338f25eeabfea5673b/users/1090776a11594f0da41468f30518a67e/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:03:58.362007 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1611b3-8b78-4c56-808c-fd70d507f047 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:58.362339 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1611b3-8b78-4c56-808c-fd70d507f047 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:58.362613 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1611b3-8b78-4c56-808c-fd70d507f047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:58.389008 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-de1611b3-8b78-4c56-808c-fd70d507f047 None None] MFA Rules not processed for user `1090776a11594f0da41468f30518a67e`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:58.390856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-de1611b3-8b78-4c56-808c-fd70d507f047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:58.453955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1147/2283] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:58.463588 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede5de5-a685-4654-a05a-287bde31fefa admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:58.532957 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede5de5-a685-4654-a05a-287bde31fefa admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:58.537901 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e702e12-cdbf-4250-947d-91bc842491e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:58.538587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e702e12-cdbf-4250-947d-91bc842491e7 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:58.538866 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e702e12-cdbf-4250-947d-91bc842491e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:58.539138 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e702e12-cdbf-4250-947d-91bc842491e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:58.539824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e702e12-cdbf-4250-947d-91bc842491e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:58.542304 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e702e12-cdbf-4250-947d-91bc842491e7 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:58.568847 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1137/2284] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:03:58 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:58.572859 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1611b3-8b78-4c56-808c-fd70d507f047 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:58.584063 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1611b3-8b78-4c56-808c-fd70d507f047 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:58.637917 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222461e-ff9e-473d-b7ea-eb5a18f6ae4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:58.638683 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222461e-ff9e-473d-b7ea-eb5a18f6ae4b admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:58.638968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222461e-ff9e-473d-b7ea-eb5a18f6ae4b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:58.639759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222461e-ff9e-473d-b7ea-eb5a18f6ae4b admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:58.640438 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f222461e-ff9e-473d-b7ea-eb5a18f6ae4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:58.642372 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f222461e-ff9e-473d-b7ea-eb5a18f6ae4b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:58.662011 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1148/2285] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:03:58 2022] POST /identity/v3/users => generated 297 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:03:58.666135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e702e12-cdbf-4250-947d-91bc842491e7 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:58.676044 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e702e12-cdbf-4250-947d-91bc842491e7 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:58.680584 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09185c8-8e8e-471b-bffb-76b1c18bee91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:58.681228 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09185c8-8e8e-471b-bffb-76b1c18bee91 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:58.681503 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09185c8-8e8e-471b-bffb-76b1c18bee91 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:58.681769 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09185c8-8e8e-471b-bffb-76b1c18bee91 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:58.682513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09185c8-8e8e-471b-bffb-76b1c18bee91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:58.684080 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09185c8-8e8e-471b-bffb-76b1c18bee91 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:58.694178 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1138/2286] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:03:58.698047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222461e-ff9e-473d-b7ea-eb5a18f6ae4b admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:58.708371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222461e-ff9e-473d-b7ea-eb5a18f6ae4b admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:58.712900 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88a06b-1308-44b1-8d3a-95f6f83c4f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:58.713819 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88a06b-1308-44b1-8d3a-95f6f83c4f84 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:58.714053 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88a06b-1308-44b1-8d3a-95f6f83c4f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:58.714359 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88a06b-1308-44b1-8d3a-95f6f83c4f84 admin admin] PATH_INFO: `/v3/projects/e5aac20604c84411b576c74e9ce18de0/users/4b16d9699d1949bda6cf1d91b721ec5b/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:58.728938 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da88a06b-1308-44b1-8d3a-95f6f83c4f84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5aac20604c84411b576c74e9ce18de0, user_id=4b16d9699d1949bda6cf1d91b721ec5b, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:58.731613 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da88a06b-1308-44b1-8d3a-95f6f83c4f84 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:58.743965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1149/2287] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:58 2022] PUT /identity/v3/projects/e5aac20604c84411b576c74e9ce18de0/users/4b16d9699d1949bda6cf1d91b721ec5b/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:58.747801 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09185c8-8e8e-471b-bffb-76b1c18bee91 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:58.757974 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09185c8-8e8e-471b-bffb-76b1c18bee91 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:58.809720 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee496f8-11b8-460d-afac-7e1d3d4fe919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:58.810440 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee496f8-11b8-460d-afac-7e1d3d4fe919 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:58.810860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee496f8-11b8-460d-afac-7e1d3d4fe919 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:58.811633 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee496f8-11b8-460d-afac-7e1d3d4fe919 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:58.812603 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee496f8-11b8-460d-afac-7e1d3d4fe919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:58.814745 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee496f8-11b8-460d-afac-7e1d3d4fe919 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:58.829719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1139/2288] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:03:58.833994 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88a06b-1308-44b1-8d3a-95f6f83c4f84 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:58.843994 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88a06b-1308-44b1-8d3a-95f6f83c4f84 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:58.848496 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c57650-fa06-424a-bd6d-d8c91a058981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:58.849497 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c57650-fa06-424a-bd6d-d8c91a058981 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:58.849774 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c57650-fa06-424a-bd6d-d8c91a058981 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:58.850040 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c57650-fa06-424a-bd6d-d8c91a058981 admin admin] PATH_INFO: `/v3/projects/e5aac20604c84411b576c74e9ce18de0/users/4b16d9699d1949bda6cf1d91b721ec5b/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:58.851986 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c57650-fa06-424a-bd6d-d8c91a058981 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5aac20604c84411b576c74e9ce18de0, user_id=4b16d9699d1949bda6cf1d91b721ec5b, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:58.854632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c57650-fa06-424a-bd6d-d8c91a058981 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:58.870091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1150/2289] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:03:58 2022] PUT /identity/v3/projects/e5aac20604c84411b576c74e9ce18de0/users/4b16d9699d1949bda6cf1d91b721ec5b/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:03:58.881350 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95779003-ef97-4428-98d7-65009868b9cb None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:58.881633 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95779003-ef97-4428-98d7-65009868b9cb None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:58.881906 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95779003-ef97-4428-98d7-65009868b9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:58.908503 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-95779003-ef97-4428-98d7-65009868b9cb None None] MFA Rules not processed for user `4b16d9699d1949bda6cf1d91b721ec5b`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:03:58.909835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-95779003-ef97-4428-98d7-65009868b9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:58.975320 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1140/2290] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:03:58 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:03:58.987169 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c57650-fa06-424a-bd6d-d8c91a058981 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:03:58.988865 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-59c57650-fa06-424a-bd6d-d8c91a058981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:03:59.046581 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c57650-fa06-424a-bd6d-d8c91a058981 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:03:59.050225 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c739d-9c87-4bc8-b92a-15cece5b6714 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:03:59.050862 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c739d-9c87-4bc8-b92a-15cece5b6714 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:03:59.051145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c739d-9c87-4bc8-b92a-15cece5b6714 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:03:59.051405 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c739d-9c87-4bc8-b92a-15cece5b6714 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:03:59.106033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c739d-9c87-4bc8-b92a-15cece5b6714 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:03:59.107751 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c739d-9c87-4bc8-b92a-15cece5b6714 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:03:59.117386 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1151/2291] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:03:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:04:01.672169 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95779003-ef97-4428-98d7-65009868b9cb None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:01.673552 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-95779003-ef97-4428-98d7-65009868b9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:01.685218 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95779003-ef97-4428-98d7-65009868b9cb None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:01.688830 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8cc235-30a9-4fec-8aaf-cc90d3d405bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:01.689466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8cc235-30a9-4fec-8aaf-cc90d3d405bf service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:01.689745 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8cc235-30a9-4fec-8aaf-cc90d3d405bf service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:01.690017 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8cc235-30a9-4fec-8aaf-cc90d3d405bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:01.748364 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8cc235-30a9-4fec-8aaf-cc90d3d405bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:01.750454 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8cc235-30a9-4fec-8aaf-cc90d3d405bf service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:01.761003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1141/2292] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:04:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:04:02.343265 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c739d-9c87-4bc8-b92a-15cece5b6714 service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:02.351878 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c739d-9c87-4bc8-b92a-15cece5b6714 service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:02.355766 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1c5ba-c38e-4b64-98ae-54ccafca4379 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:02.356405 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1c5ba-c38e-4b64-98ae-54ccafca4379 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:02.356686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1c5ba-c38e-4b64-98ae-54ccafca4379 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:02.357229 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1c5ba-c38e-4b64-98ae-54ccafca4379 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:02.367124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1c5ba-c38e-4b64-98ae-54ccafca4379 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:02.368542 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1c5ba-c38e-4b64-98ae-54ccafca4379 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:02.378396 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1152/2293] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:04:02 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:04:08.132548 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8cc235-30a9-4fec-8aaf-cc90d3d405bf service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:08.140348 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ef8cc235-30a9-4fec-8aaf-cc90d3d405bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:08.253454 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8cc235-30a9-4fec-8aaf-cc90d3d405bf service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:08.258087 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304720b-60cc-49ca-8fae-6bcfbec958cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:08.259116 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304720b-60cc-49ca-8fae-6bcfbec958cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:08.259391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304720b-60cc-49ca-8fae-6bcfbec958cf admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:08.259658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304720b-60cc-49ca-8fae-6bcfbec958cf admin admin] PATH_INFO: `/v3/users/ed3226d78e4248839447d36e4694d997` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:08.260841 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304720b-60cc-49ca-8fae-6bcfbec958cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed3226d78e4248839447d36e4694d997)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:08.263038 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304720b-60cc-49ca-8fae-6bcfbec958cf admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:08.316877 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e304720b-60cc-49ca-8fae-6bcfbec958cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed3226d78e4248839447d36e4694d997'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:08.337038 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e304720b-60cc-49ca-8fae-6bcfbec958cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed3226d78e4248839447d36e4694d997'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:08.338943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e304720b-60cc-49ca-8fae-6bcfbec958cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed3226d78e4248839447d36e4694d997'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:08.353534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e304720b-60cc-49ca-8fae-6bcfbec958cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed3226d78e4248839447d36e4694d997'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:08.354330 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e304720b-60cc-49ca-8fae-6bcfbec958cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed3226d78e4248839447d36e4694d997'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:08.370925 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1c5ba-c38e-4b64-98ae-54ccafca4379 service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:08.381521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1c5ba-c38e-4b64-98ae-54ccafca4379 service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:08.385893 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1142/2294] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:04:08 2022] DELETE /identity/v3/users/ed3226d78e4248839447d36e4694d997 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:04:08.390519 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf0657-56ce-42df-a0ab-8db859062032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:08.391186 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5868060-e902-4a10-bf5e-146622a1d30f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:08.391577 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5868060-e902-4a10-bf5e-146622a1d30f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:08.391874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf0657-56ce-42df-a0ab-8db859062032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:08.392005 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5868060-e902-4a10-bf5e-146622a1d30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:08.392458 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf0657-56ce-42df-a0ab-8db859062032 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:08.392871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf0657-56ce-42df-a0ab-8db859062032 admin admin] PATH_INFO: `/v3/users/695871ce6a9c409c868d52cff37b090f` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:08.394358 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf0657-56ce-42df-a0ab-8db859062032 admin admin] RBAC: Authorizing `identity:delete_user(user_id=695871ce6a9c409c868d52cff37b090f)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:08.396492 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf0657-56ce-42df-a0ab-8db859062032 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:08.412710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-f5868060-e902-4a10-bf5e-146622a1d30f None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:08.414383 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f5868060-e902-4a10-bf5e-146622a1d30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:08.456998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-5ebf0657-56ce-42df-a0ab-8db859062032 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '695871ce6a9c409c868d52cff37b090f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:08.466315 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-5ebf0657-56ce-42df-a0ab-8db859062032 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '695871ce6a9c409c868d52cff37b090f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:08.476585 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-5ebf0657-56ce-42df-a0ab-8db859062032 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '695871ce6a9c409c868d52cff37b090f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:08.477756 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-5ebf0657-56ce-42df-a0ab-8db859062032 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '695871ce6a9c409c868d52cff37b090f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:08.499741 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-5ebf0657-56ce-42df-a0ab-8db859062032 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '695871ce6a9c409c868d52cff37b090f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:08.500579 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1143/2295] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:04:08.511490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1153/2296] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:04:08 2022] DELETE /identity/v3/users/695871ce6a9c409c868d52cff37b090f => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:04:08.516391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ee15a-0438-4a79-8270-6c05f2fefc05 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:08.516820 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ee15a-0438-4a79-8270-6c05f2fefc05 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:08.517232 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ee15a-0438-4a79-8270-6c05f2fefc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:08.535989 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-469ee15a-0438-4a79-8270-6c05f2fefc05 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:08.537195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-469ee15a-0438-4a79-8270-6c05f2fefc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:08.550240 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf0657-56ce-42df-a0ab-8db859062032 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:08.552116 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5ebf0657-56ce-42df-a0ab-8db859062032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:08.616258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1144/2297] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:08.646008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ee15a-0438-4a79-8270-6c05f2fefc05 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:08.646008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-469ee15a-0438-4a79-8270-6c05f2fefc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:08.672963 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf0657-56ce-42df-a0ab-8db859062032 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:08.678062 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc02a61-cbb4-46e1-a307-93cc34d111e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:08.678921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc02a61-cbb4-46e1-a307-93cc34d111e6 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:08.679555 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc02a61-cbb4-46e1-a307-93cc34d111e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:08.680545 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc02a61-cbb4-46e1-a307-93cc34d111e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:08.683352 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6bc02a61-cbb4-46e1-a307-93cc34d111e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:08.705890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc02a61-cbb4-46e1-a307-93cc34d111e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:08.707773 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc02a61-cbb4-46e1-a307-93cc34d111e6 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:08.720160 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1154/2298] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:04:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:04:08.739125 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ee15a-0438-4a79-8270-6c05f2fefc05 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:08.747086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3f949-cfd4-4c97-928c-300f4fa6f0cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:08.749020 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3f949-cfd4-4c97-928c-300f4fa6f0cc service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:08.749527 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3f949-cfd4-4c97-928c-300f4fa6f0cc service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:08.750014 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3f949-cfd4-4c97-928c-300f4fa6f0cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:08.765107 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3f949-cfd4-4c97-928c-300f4fa6f0cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:08.767369 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3f949-cfd4-4c97-928c-300f4fa6f0cc service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:08.784010 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1145/2299] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:04:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:04:08.810891 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db328012-5f11-49e5-a575-d6fa74dc429b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:08.810891 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db328012-5f11-49e5-a575-d6fa74dc429b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:08.811718 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db328012-5f11-49e5-a575-d6fa74dc429b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:08.833136 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-db328012-5f11-49e5-a575-d6fa74dc429b None None] MFA Rules not processed for user `1a6f50dcebf146baae8a9f0b5b112795`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:08.834562 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-db328012-5f11-49e5-a575-d6fa74dc429b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:08.898989 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1155/2300] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 9 23:04:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:04:08.953256 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3f949-cfd4-4c97-928c-300f4fa6f0cc service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:08.955983 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ccd3f949-cfd4-4c97-928c-300f4fa6f0cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:09.042470 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3f949-cfd4-4c97-928c-300f4fa6f0cc service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:09.048559 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c76b77-ab3a-4a04-9a75-cb8e5425994a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cb6c294ce1842b68a6bc73ac381ed6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:09.049283 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c76b77-ab3a-4a04-9a75-cb8e5425994a service swift] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:09.049657 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c76b77-ab3a-4a04-9a75-cb8e5425994a service swift] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:09.050086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c76b77-ab3a-4a04-9a75-cb8e5425994a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:09.061693 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c76b77-ab3a-4a04-9a75-cb8e5425994a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:09.063803 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c76b77-ab3a-4a04-9a75-cb8e5425994a service swift] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:09.075152 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1146/2301] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 9 23:04:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 23:04:09.147082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db328012-5f11-49e5-a575-d6fa74dc429b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:09.148753 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-db328012-5f11-49e5-a575-d6fa74dc429b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:09.166490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db328012-5f11-49e5-a575-d6fa74dc429b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:09.173504 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f2059-6b4c-4a6f-9ee3-47e835015fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:09.174541 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f2059-6b4c-4a6f-9ee3-47e835015fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:09.176545 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f2059-6b4c-4a6f-9ee3-47e835015fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:09.176545 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f2059-6b4c-4a6f-9ee3-47e835015fbc admin admin] PATH_INFO: `/v3/projects/d73ac74cb24d432ba9e5496e7c0e291e` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:09.176545 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f2059-6b4c-4a6f-9ee3-47e835015fbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d73ac74cb24d432ba9e5496e7c0e291e)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:09.179349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f2059-6b4c-4a6f-9ee3-47e835015fbc admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:09.281589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-500f2059-6b4c-4a6f-9ee3-47e835015fbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd73ac74cb24d432ba9e5496e7c0e291e', 'request_id': 'req-500f2059-6b4c-4a6f-9ee3-47e835015fbc'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:09.287655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-500f2059-6b4c-4a6f-9ee3-47e835015fbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd73ac74cb24d432ba9e5496e7c0e291e', 'request_id': 'req-500f2059-6b4c-4a6f-9ee3-47e835015fbc'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:09.298516 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-500f2059-6b4c-4a6f-9ee3-47e835015fbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd73ac74cb24d432ba9e5496e7c0e291e', 'request_id': 'req-500f2059-6b4c-4a6f-9ee3-47e835015fbc'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:09.317241 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-500f2059-6b4c-4a6f-9ee3-47e835015fbc admin admin] The token cache is being invalidate because project d73ac74cb24d432ba9e5496e7c0e291e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:04:09.318341 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-500f2059-6b4c-4a6f-9ee3-47e835015fbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:09.320120 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-500f2059-6b4c-4a6f-9ee3-47e835015fbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:09.322215 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1156/2302] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:04:09 2022] DELETE /identity/v3/projects/d73ac74cb24d432ba9e5496e7c0e291e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:04:09.331059 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7f9e8-17de-4260-89e9-a8a4d2aa28a6 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:09.331059 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7f9e8-17de-4260-89e9-a8a4d2aa28a6 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:09.331059 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7f9e8-17de-4260-89e9-a8a4d2aa28a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:09.354734 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-88d7f9e8-17de-4260-89e9-a8a4d2aa28a6 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:09.363837 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-88d7f9e8-17de-4260-89e9-a8a4d2aa28a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:09.446151 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1147/2303] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:09.452062 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e60e3-c361-4c20-aa4d-90b25af78c76 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:09.452931 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e60e3-c361-4c20-aa4d-90b25af78c76 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:09.453389 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e60e3-c361-4c20-aa4d-90b25af78c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:09.477012 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-db7e60e3-c361-4c20-aa4d-90b25af78c76 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:09.478317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-db7e60e3-c361-4c20-aa4d-90b25af78c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:09.487536 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1157/2304] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:09.491848 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7f9e8-17de-4260-89e9-a8a4d2aa28a6 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:09.501474 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7f9e8-17de-4260-89e9-a8a4d2aa28a6 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:09.506300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d2675-8815-40e8-ad9c-3550dd2d3227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:09.506955 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d2675-8815-40e8-ad9c-3550dd2d3227 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:09.507330 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d2675-8815-40e8-ad9c-3550dd2d3227 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:09.507772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d2675-8815-40e8-ad9c-3550dd2d3227 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:09.508500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d2675-8815-40e8-ad9c-3550dd2d3227 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:04:09.508801 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d2675-8815-40e8-ad9c-3550dd2d3227 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:09.510660 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d2675-8815-40e8-ad9c-3550dd2d3227 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:09.523386 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e60e3-c361-4c20-aa4d-90b25af78c76 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:09.525158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1148/2305] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:04:09.525834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-db7e60e3-c361-4c20-aa4d-90b25af78c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:09.530342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d2675-8815-40e8-ad9c-3550dd2d3227 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:09.547508 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d2675-8815-40e8-ad9c-3550dd2d3227 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:09.554414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e60e3-c361-4c20-aa4d-90b25af78c76 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:09.559496 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9583eca-6087-4cd7-a818-c45e4adb6162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:09.560385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9583eca-6087-4cd7-a818-c45e4adb6162 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:09.560819 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9583eca-6087-4cd7-a818-c45e4adb6162 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:09.561282 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9583eca-6087-4cd7-a818-c45e4adb6162 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:09.562179 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9583eca-6087-4cd7-a818-c45e4adb6162 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:09.564900 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9583eca-6087-4cd7-a818-c45e4adb6162 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:09.567228 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086fac9-1436-4f15-8c84-f82edbdd40b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:09.568093 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086fac9-1436-4f15-8c84-f82edbdd40b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:09.568373 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086fac9-1436-4f15-8c84-f82edbdd40b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:09.568700 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086fac9-1436-4f15-8c84-f82edbdd40b9 admin admin] PATH_INFO: `/v3/projects/e046fbe4d4c24b429f85f8eee72423bf` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:09.573447 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a086fac9-1436-4f15-8c84-f82edbdd40b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e046fbe4d4c24b429f85f8eee72423bf)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:09.590341 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a086fac9-1436-4f15-8c84-f82edbdd40b9 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:09.603686 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1149/2306] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:04:09 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:04:09.607551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9583eca-6087-4cd7-a818-c45e4adb6162 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:09.666583 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a086fac9-1436-4f15-8c84-f82edbdd40b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e046fbe4d4c24b429f85f8eee72423bf', 'request_id': 'req-a086fac9-1436-4f15-8c84-f82edbdd40b9'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:09.673005 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a086fac9-1436-4f15-8c84-f82edbdd40b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e046fbe4d4c24b429f85f8eee72423bf', 'request_id': 'req-a086fac9-1436-4f15-8c84-f82edbdd40b9'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:09.679911 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a086fac9-1436-4f15-8c84-f82edbdd40b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e046fbe4d4c24b429f85f8eee72423bf', 'request_id': 'req-a086fac9-1436-4f15-8c84-f82edbdd40b9'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:09.690570 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9583eca-6087-4cd7-a818-c45e4adb6162 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:09.695645 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a086fac9-1436-4f15-8c84-f82edbdd40b9 admin admin] The token cache is being invalidate because project e046fbe4d4c24b429f85f8eee72423bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:04:09.695944 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a086fac9-1436-4f15-8c84-f82edbdd40b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:09.696641 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a086fac9-1436-4f15-8c84-f82edbdd40b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:09.701322 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1158/2307] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:04:09 2022] DELETE /identity/v3/projects/e046fbe4d4c24b429f85f8eee72423bf => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:04:09.702866 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086fac9-1436-4f15-8c84-f82edbdd40b9 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:09.704258 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a086fac9-1436-4f15-8c84-f82edbdd40b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:09.765995 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbc187-d41d-4eee-acfd-96f07dd3cdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:09.766719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbc187-d41d-4eee-acfd-96f07dd3cdd0 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:09.767003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbc187-d41d-4eee-acfd-96f07dd3cdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:09.767276 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbc187-d41d-4eee-acfd-96f07dd3cdd0 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:09.767969 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dbc187-d41d-4eee-acfd-96f07dd3cdd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:09.769885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dbc187-d41d-4eee-acfd-96f07dd3cdd0 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:09.782292 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086fac9-1436-4f15-8c84-f82edbdd40b9 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:09.788534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f006def-2d7c-460d-9e08-59039d46c7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:09.789758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f006def-2d7c-460d-9e08-59039d46c7d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:09.790042 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f006def-2d7c-460d-9e08-59039d46c7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:09.790465 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f006def-2d7c-460d-9e08-59039d46c7d7 admin admin] PATH_INFO: `/v3/users/1090776a11594f0da41468f30518a67e` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:09.792007 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f006def-2d7c-460d-9e08-59039d46c7d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1090776a11594f0da41468f30518a67e)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:09.798054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f006def-2d7c-460d-9e08-59039d46c7d7 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:09.798468 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1150/2308] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:04:09 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:04:09.799609 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbc187-d41d-4eee-acfd-96f07dd3cdd0 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:09.801952 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-62dbc187-d41d-4eee-acfd-96f07dd3cdd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:09.819313 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbc187-d41d-4eee-acfd-96f07dd3cdd0 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:09.831332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0c654-4ccf-4d10-b1cb-7b225d316101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:09.832368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0c654-4ccf-4d10-b1cb-7b225d316101 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:09.833145 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0c654-4ccf-4d10-b1cb-7b225d316101 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:09.833678 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0c654-4ccf-4d10-b1cb-7b225d316101 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:09.835821 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0c654-4ccf-4d10-b1cb-7b225d316101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:09.837861 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0c654-4ccf-4d10-b1cb-7b225d316101 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:09.865877 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6f006def-2d7c-460d-9e08-59039d46c7d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1090776a11594f0da41468f30518a67e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:09.876521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6f006def-2d7c-460d-9e08-59039d46c7d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1090776a11594f0da41468f30518a67e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:09.882794 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1151/2309] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:04:09.886813 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6f006def-2d7c-460d-9e08-59039d46c7d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1090776a11594f0da41468f30518a67e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:09.887129 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0c654-4ccf-4d10-b1cb-7b225d316101 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:09.888028 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6f006def-2d7c-460d-9e08-59039d46c7d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1090776a11594f0da41468f30518a67e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:09.899457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0c654-4ccf-4d10-b1cb-7b225d316101 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:09.903039 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6f006def-2d7c-460d-9e08-59039d46c7d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1090776a11594f0da41468f30518a67e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:09.905601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92120b13-aba1-4b07-bc37-f959bb7b6c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:09.907078 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92120b13-aba1-4b07-bc37-f959bb7b6c02 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:09.907078 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92120b13-aba1-4b07-bc37-f959bb7b6c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:09.907078 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92120b13-aba1-4b07-bc37-f959bb7b6c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:09.907910 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92120b13-aba1-4b07-bc37-f959bb7b6c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:09.909588 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92120b13-aba1-4b07-bc37-f959bb7b6c02 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:09.912825 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1159/2310] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:04:09 2022] DELETE /identity/v3/users/1090776a11594f0da41468f30518a67e => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:04:09.912825 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff21988-611b-4cdf-a5fc-f3aa985cc483 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:09.912825 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff21988-611b-4cdf-a5fc-f3aa985cc483 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:09.912825 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff21988-611b-4cdf-a5fc-f3aa985cc483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:09.935838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-7ff21988-611b-4cdf-a5fc-f3aa985cc483 None None] MFA Rules not processed for user `1a6f50dcebf146baae8a9f0b5b112795`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:09.940067 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1152/2311] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:04:09.943535 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7ff21988-611b-4cdf-a5fc-f3aa985cc483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:09.947948 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92120b13-aba1-4b07-bc37-f959bb7b6c02 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:09.952506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-92120b13-aba1-4b07-bc37-f959bb7b6c02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:09.968708 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92120b13-aba1-4b07-bc37-f959bb7b6c02 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:10.021387 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1160/2312] 10.222.0.23 () {58 vars in 967 bytes} [Mon May 9 23:04:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:10.044169 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff21988-611b-4cdf-a5fc-f3aa985cc483 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:10.046156 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7ff21988-611b-4cdf-a5fc-f3aa985cc483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:10.047872 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acdbce3-0a45-4425-823c-02263f107e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:10.049397 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acdbce3-0a45-4425-823c-02263f107e10 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:10.050067 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acdbce3-0a45-4425-823c-02263f107e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:10.051169 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acdbce3-0a45-4425-823c-02263f107e10 admin admin] PATH_INFO: `/v3/projects/93df4f41ba52453dabfd21f82cb9238b/users/19fbe84526dd48a6be14c1aebf5713c0/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:10.070927 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acdbce3-0a45-4425-823c-02263f107e10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93df4f41ba52453dabfd21f82cb9238b, user_id=19fbe84526dd48a6be14c1aebf5713c0, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:10.074563 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acdbce3-0a45-4425-823c-02263f107e10 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:10.100181 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1153/2313] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:04:09 2022] PUT /identity/v3/projects/93df4f41ba52453dabfd21f82cb9238b/users/19fbe84526dd48a6be14c1aebf5713c0/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:04:10.107302 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b31ec2-67e1-4f9d-9eaa-129f7673ff85 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:10.108243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b31ec2-67e1-4f9d-9eaa-129f7673ff85 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:10.108881 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b31ec2-67e1-4f9d-9eaa-129f7673ff85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:10.133197 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff21988-611b-4cdf-a5fc-f3aa985cc483 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:10.135930 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-70b31ec2-67e1-4f9d-9eaa-129f7673ff85 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:10.137139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-70b31ec2-67e1-4f9d-9eaa-129f7673ff85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:10.226855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff16fe1-196a-4082-b11c-8ca8cf804cae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cb6c294ce1842b68a6bc73ac381ed6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:10.228076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff16fe1-196a-4082-b11c-8ca8cf804cae service swift] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:10.228904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff16fe1-196a-4082-b11c-8ca8cf804cae service swift] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:10.229999 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff16fe1-196a-4082-b11c-8ca8cf804cae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:10.233632 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1154/2314] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:10.250482 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b31ec2-67e1-4f9d-9eaa-129f7673ff85 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:10.252248 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-70b31ec2-67e1-4f9d-9eaa-129f7673ff85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:10.341927 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff16fe1-196a-4082-b11c-8ca8cf804cae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:10.343722 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff16fe1-196a-4082-b11c-8ca8cf804cae service swift] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:10.353720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1161/2315] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 9 23:04:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 23:04:10.361735 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b31ec2-67e1-4f9d-9eaa-129f7673ff85 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:10.370307 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed71ec-13cb-48c6-a118-865e551aeaff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:10.374844 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed71ec-13cb-48c6-a118-865e551aeaff service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:10.375128 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed71ec-13cb-48c6-a118-865e551aeaff service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:10.375404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed71ec-13cb-48c6-a118-865e551aeaff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:10.396911 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed71ec-13cb-48c6-a118-865e551aeaff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:10.398372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed71ec-13cb-48c6-a118-865e551aeaff service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:10.410329 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1155/2316] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:04:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:04:10.442631 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff16fe1-196a-4082-b11c-8ca8cf804cae service swift] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:10.444289 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1ff16fe1-196a-4082-b11c-8ca8cf804cae service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:10.486156 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff16fe1-196a-4082-b11c-8ca8cf804cae service swift] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:10.500738 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c05410-341b-4abc-ba0d-dbbf10311b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:10.502309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c05410-341b-4abc-ba0d-dbbf10311b71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:10.503699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c05410-341b-4abc-ba0d-dbbf10311b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:10.505918 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c05410-341b-4abc-ba0d-dbbf10311b71 admin admin] PATH_INFO: `/v3/projects/89be2e7ed3654f338f25eeabfea5673b` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:10.507832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c05410-341b-4abc-ba0d-dbbf10311b71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89be2e7ed3654f338f25eeabfea5673b)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:10.510753 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c05410-341b-4abc-ba0d-dbbf10311b71 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:10.640049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-63c05410-341b-4abc-ba0d-dbbf10311b71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89be2e7ed3654f338f25eeabfea5673b', 'request_id': 'req-63c05410-341b-4abc-ba0d-dbbf10311b71'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:10.648404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-63c05410-341b-4abc-ba0d-dbbf10311b71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89be2e7ed3654f338f25eeabfea5673b', 'request_id': 'req-63c05410-341b-4abc-ba0d-dbbf10311b71'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:10.661196 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-63c05410-341b-4abc-ba0d-dbbf10311b71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89be2e7ed3654f338f25eeabfea5673b', 'request_id': 'req-63c05410-341b-4abc-ba0d-dbbf10311b71'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:10.689981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-63c05410-341b-4abc-ba0d-dbbf10311b71 admin admin] The token cache is being invalidate because project 89be2e7ed3654f338f25eeabfea5673b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:04:10.691611 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-63c05410-341b-4abc-ba0d-dbbf10311b71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:10.693215 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-63c05410-341b-4abc-ba0d-dbbf10311b71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:10.702346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1162/2317] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:04:10 2022] DELETE /identity/v3/projects/89be2e7ed3654f338f25eeabfea5673b => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:04:10.702565 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed71ec-13cb-48c6-a118-865e551aeaff service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:10.716894 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed71ec-13cb-48c6-a118-865e551aeaff service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:10.804091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca33be3-5f48-4063-9d23-4bebf27c3991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:10.805018 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca33be3-5f48-4063-9d23-4bebf27c3991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:10.805723 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca33be3-5f48-4063-9d23-4bebf27c3991 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:10.806015 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca33be3-5f48-4063-9d23-4bebf27c3991 admin admin] PATH_INFO: `/v3/users/4b16d9699d1949bda6cf1d91b721ec5b` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:10.807769 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca33be3-5f48-4063-9d23-4bebf27c3991 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b16d9699d1949bda6cf1d91b721ec5b)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:10.810906 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca33be3-5f48-4063-9d23-4bebf27c3991 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:10.910893 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-6ca33be3-5f48-4063-9d23-4bebf27c3991 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b16d9699d1949bda6cf1d91b721ec5b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:10.921178 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-6ca33be3-5f48-4063-9d23-4bebf27c3991 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b16d9699d1949bda6cf1d91b721ec5b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:10.922571 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-6ca33be3-5f48-4063-9d23-4bebf27c3991 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b16d9699d1949bda6cf1d91b721ec5b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:10.932174 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951f10a-b900-403f-b558-f83f23c24e6d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:10.932387 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951f10a-b900-403f-b558-f83f23c24e6d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:10.932658 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951f10a-b900-403f-b558-f83f23c24e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:10.939561 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-6ca33be3-5f48-4063-9d23-4bebf27c3991 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b16d9699d1949bda6cf1d91b721ec5b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:10.941171 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-6ca33be3-5f48-4063-9d23-4bebf27c3991 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b16d9699d1949bda6cf1d91b721ec5b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:10.954629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5951f10a-b900-403f-b558-f83f23c24e6d None None] MFA Rules not processed for user `1a6f50dcebf146baae8a9f0b5b112795`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:10.956313 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5951f10a-b900-403f-b558-f83f23c24e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:10.964888 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1156/2318] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:04:10 2022] DELETE /identity/v3/users/4b16d9699d1949bda6cf1d91b721ec5b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:04:11.034042 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1163/2319] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 9 23:04:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:11.047538 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca33be3-5f48-4063-9d23-4bebf27c3991 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:11.049090 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-6ca33be3-5f48-4063-9d23-4bebf27c3991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:11.143604 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca33be3-5f48-4063-9d23-4bebf27c3991 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:11.150275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398e596-7910-4118-9e31-94acdd5f1aa5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cb6c294ce1842b68a6bc73ac381ed6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:11.151233 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398e596-7910-4118-9e31-94acdd5f1aa5 service swift] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:11.151704 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398e596-7910-4118-9e31-94acdd5f1aa5 service swift] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:11.152162 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398e596-7910-4118-9e31-94acdd5f1aa5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:11.235325 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2398e596-7910-4118-9e31-94acdd5f1aa5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:11.237537 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2398e596-7910-4118-9e31-94acdd5f1aa5 service swift] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:11.263183 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1157/2320] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 9 23:04:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 23:04:11.375795 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951f10a-b900-403f-b558-f83f23c24e6d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:11.378015 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5951f10a-b900-403f-b558-f83f23c24e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:11.401009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951f10a-b900-403f-b558-f83f23c24e6d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:11.484513 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91998298-5153-40ef-8b8c-9d33cec04bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:11.485548 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91998298-5153-40ef-8b8c-9d33cec04bc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:11.486184 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91998298-5153-40ef-8b8c-9d33cec04bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:11.486727 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91998298-5153-40ef-8b8c-9d33cec04bc5 admin admin] PATH_INFO: `/v3/projects/e5aac20604c84411b576c74e9ce18de0` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:11.488559 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91998298-5153-40ef-8b8c-9d33cec04bc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5aac20604c84411b576c74e9ce18de0)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:11.490885 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91998298-5153-40ef-8b8c-9d33cec04bc5 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:11.555722 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-91998298-5153-40ef-8b8c-9d33cec04bc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5aac20604c84411b576c74e9ce18de0', 'request_id': 'req-91998298-5153-40ef-8b8c-9d33cec04bc5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:11.561744 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-91998298-5153-40ef-8b8c-9d33cec04bc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5aac20604c84411b576c74e9ce18de0', 'request_id': 'req-91998298-5153-40ef-8b8c-9d33cec04bc5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:11.568886 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-91998298-5153-40ef-8b8c-9d33cec04bc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5aac20604c84411b576c74e9ce18de0', 'request_id': 'req-91998298-5153-40ef-8b8c-9d33cec04bc5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:11.587379 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-91998298-5153-40ef-8b8c-9d33cec04bc5 admin admin] The token cache is being invalidate because project e5aac20604c84411b576c74e9ce18de0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:04:11.587669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-91998298-5153-40ef-8b8c-9d33cec04bc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:11.588580 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-91998298-5153-40ef-8b8c-9d33cec04bc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:11.592026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1164/2321] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:04:11 2022] DELETE /identity/v3/projects/e5aac20604c84411b576c74e9ce18de0 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:04:11.596670 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8564e1-df25-4225-bf2e-e917225edeff None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:11.597496 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8564e1-df25-4225-bf2e-e917225edeff None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:11.597764 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8564e1-df25-4225-bf2e-e917225edeff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:11.622640 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-3d8564e1-df25-4225-bf2e-e917225edeff None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:11.624723 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3d8564e1-df25-4225-bf2e-e917225edeff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:11.689907 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66b9a1-6038-4ea8-a5c6-31bb5d6e5aa1 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:11.690431 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66b9a1-6038-4ea8-a5c6-31bb5d6e5aa1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:11.690718 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66b9a1-6038-4ea8-a5c6-31bb5d6e5aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:11.715790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-da66b9a1-6038-4ea8-a5c6-31bb5d6e5aa1 None None] MFA Rules not processed for user `1a6f50dcebf146baae8a9f0b5b112795`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:11.717016 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-da66b9a1-6038-4ea8-a5c6-31bb5d6e5aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:11.725415 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1158/2322] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:11.733894 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27268c9-9433-4eb6-bf94-c5e05a6c7975 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:11.734213 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27268c9-9433-4eb6-bf94-c5e05a6c7975 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:11.734499 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27268c9-9433-4eb6-bf94-c5e05a6c7975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:11.758910 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a27268c9-9433-4eb6-bf94-c5e05a6c7975 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:11.760686 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a27268c9-9433-4eb6-bf94-c5e05a6c7975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:11.774605 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1159/2323] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:11.778462 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27268c9-9433-4eb6-bf94-c5e05a6c7975 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:11.796010 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27268c9-9433-4eb6-bf94-c5e05a6c7975 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:11.807434 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52039481-c55b-47e2-a896-4a5248054a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:11.808218 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52039481-c55b-47e2-a896-4a5248054a1e admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:11.808426 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52039481-c55b-47e2-a896-4a5248054a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:11.808708 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52039481-c55b-47e2-a896-4a5248054a1e admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:11.809890 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52039481-c55b-47e2-a896-4a5248054a1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:04:11.810226 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52039481-c55b-47e2-a896-4a5248054a1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:11.817262 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52039481-c55b-47e2-a896-4a5248054a1e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:11.837185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1165/2324] 10.222.0.23 () {58 vars in 967 bytes} [Mon May 9 23:04:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:04:11.851687 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da66b9a1-6038-4ea8-a5c6-31bb5d6e5aa1 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:11.855049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-da66b9a1-6038-4ea8-a5c6-31bb5d6e5aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:11.861109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1160/2325] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 23:04:11.865490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52039481-c55b-47e2-a896-4a5248054a1e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:11.881283 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52039481-c55b-47e2-a896-4a5248054a1e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:11.889496 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d5bfb-6409-43d7-801f-6affc1f6468d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:11.890175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d5bfb-6409-43d7-801f-6affc1f6468d admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:11.890455 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d5bfb-6409-43d7-801f-6affc1f6468d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:11.890722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d5bfb-6409-43d7-801f-6affc1f6468d admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:11.891754 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d5bfb-6409-43d7-801f-6affc1f6468d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:11.894058 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d5bfb-6409-43d7-801f-6affc1f6468d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:11.930161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1161/2326] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:04:11 2022] POST /identity/v3/projects => generated 399 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:04:11.932492 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d5bfb-6409-43d7-801f-6affc1f6468d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:11.961847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da66b9a1-6038-4ea8-a5c6-31bb5d6e5aa1 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:12.024953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d5bfb-6409-43d7-801f-6affc1f6468d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:12.033838 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14239a70-31a0-488f-be93-dbc1353f3dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:12.035917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14239a70-31a0-488f-be93-dbc1353f3dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:12.037095 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14239a70-31a0-488f-be93-dbc1353f3dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:12.037792 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14239a70-31a0-488f-be93-dbc1353f3dd0 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:12.039372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14239a70-31a0-488f-be93-dbc1353f3dd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:12.041729 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14239a70-31a0-488f-be93-dbc1353f3dd0 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:12.068589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cfab13-fb3a-4c37-b389-92183fbbfece service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cb6c294ce1842b68a6bc73ac381ed6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:12.069259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfab13-fb3a-4c37-b389-92183fbbfece service swift] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:12.069604 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfab13-fb3a-4c37-b389-92183fbbfece service swift] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:12.069958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfab13-fb3a-4c37-b389-92183fbbfece service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:12.070675 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1162/2327] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:04:11 2022] POST /identity/v3/users => generated 300 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:04:12.074522 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14239a70-31a0-488f-be93-dbc1353f3dd0 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:12.090151 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14239a70-31a0-488f-be93-dbc1353f3dd0 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:12.096682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c171894-a43e-408c-b483-39e11481ed85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:12.097805 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c171894-a43e-408c-b483-39e11481ed85 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:12.098347 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c171894-a43e-408c-b483-39e11481ed85 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:12.098792 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c171894-a43e-408c-b483-39e11481ed85 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:12.099727 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c171894-a43e-408c-b483-39e11481ed85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:12.103702 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c171894-a43e-408c-b483-39e11481ed85 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:12.117811 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1163/2328] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:04:12.121450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c171894-a43e-408c-b483-39e11481ed85 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:12.136484 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c171894-a43e-408c-b483-39e11481ed85 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:12.142537 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf1717-1bee-4313-9d9b-ee4983de3bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:12.143875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf1717-1bee-4313-9d9b-ee4983de3bcf admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:12.143875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf1717-1bee-4313-9d9b-ee4983de3bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:12.144211 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf1717-1bee-4313-9d9b-ee4983de3bcf admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:12.145060 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf1717-1bee-4313-9d9b-ee4983de3bcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:12.146925 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf1717-1bee-4313-9d9b-ee4983de3bcf admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:12.152973 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cfab13-fb3a-4c37-b389-92183fbbfece service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:12.155678 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cfab13-fb3a-4c37-b389-92183fbbfece service swift] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:12.158895 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1164/2329] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:04:12.162946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf1717-1bee-4313-9d9b-ee4983de3bcf admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:12.166413 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1166/2330] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 9 23:04:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 23:04:12.178799 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf1717-1bee-4313-9d9b-ee4983de3bcf admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:12.185147 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbe2f0-c3c8-479e-91d2-82f98893309d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:12.186648 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbe2f0-c3c8-479e-91d2-82f98893309d admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:12.186941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbe2f0-c3c8-479e-91d2-82f98893309d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:12.187246 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbe2f0-c3c8-479e-91d2-82f98893309d admin admin] PATH_INFO: `/v3/projects/cf71a634012b41eb9e7540df3b1dddb3/users/4546c9ee47ce475bbf2aaad9794dcfa1/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:12.223858 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fbe2f0-c3c8-479e-91d2-82f98893309d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf71a634012b41eb9e7540df3b1dddb3, user_id=4546c9ee47ce475bbf2aaad9794dcfa1, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:12.226461 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fbe2f0-c3c8-479e-91d2-82f98893309d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:12.252637 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1165/2331] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:04:12 2022] PUT /identity/v3/projects/cf71a634012b41eb9e7540df3b1dddb3/users/4546c9ee47ce475bbf2aaad9794dcfa1/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:04:12.329415 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db1618-1565-4751-91b7-58289f04310d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:12.329692 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db1618-1565-4751-91b7-58289f04310d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:12.329967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db1618-1565-4751-91b7-58289f04310d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:12.373113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-98db1618-1565-4751-91b7-58289f04310d None None] MFA Rules not processed for user `4546c9ee47ce475bbf2aaad9794dcfa1`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:12.375418 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-98db1618-1565-4751-91b7-58289f04310d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:12.468883 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1167/2332] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:12 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:12.491186 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbe2f0-c3c8-479e-91d2-82f98893309d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:12.493659 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d6fbe2f0-c3c8-479e-91d2-82f98893309d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:12.618597 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbe2f0-c3c8-479e-91d2-82f98893309d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:12.631897 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447599a-1295-468f-b9cd-b16841a9dcf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:12.632754 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447599a-1295-468f-b9cd-b16841a9dcf8 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:12.633037 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447599a-1295-468f-b9cd-b16841a9dcf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:12.633310 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447599a-1295-468f-b9cd-b16841a9dcf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:12.644700 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447599a-1295-468f-b9cd-b16841a9dcf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:12.646276 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447599a-1295-468f-b9cd-b16841a9dcf8 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:12.658481 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1166/2333] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:04:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:04:12.870042 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db1618-1565-4751-91b7-58289f04310d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:12.873934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-98db1618-1565-4751-91b7-58289f04310d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:12.996230 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db1618-1565-4751-91b7-58289f04310d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:13.001494 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db74487-7bb6-4914-843c-439edfc11c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:13.002395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db74487-7bb6-4914-843c-439edfc11c45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:13.003301 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db74487-7bb6-4914-843c-439edfc11c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:13.003584 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db74487-7bb6-4914-843c-439edfc11c45 admin admin] PATH_INFO: `/v3/users/c9a9418ba4d643e5922979677d757cf7` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:13.005847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db74487-7bb6-4914-843c-439edfc11c45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9a9418ba4d643e5922979677d757cf7)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:13.007121 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db74487-7bb6-4914-843c-439edfc11c45 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:13.065244 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7db74487-7bb6-4914-843c-439edfc11c45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9a9418ba4d643e5922979677d757cf7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:13.074711 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7db74487-7bb6-4914-843c-439edfc11c45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9a9418ba4d643e5922979677d757cf7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:13.084797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7db74487-7bb6-4914-843c-439edfc11c45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9a9418ba4d643e5922979677d757cf7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:13.085540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7db74487-7bb6-4914-843c-439edfc11c45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9a9418ba4d643e5922979677d757cf7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:13.109746 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7db74487-7bb6-4914-843c-439edfc11c45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9a9418ba4d643e5922979677d757cf7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:13.124710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1168/2334] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:04:12 2022] DELETE /identity/v3/users/c9a9418ba4d643e5922979677d757cf7 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:04:13.127079 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459a633-deea-4a6a-8213-1ceb08a23358 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:13.127445 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459a633-deea-4a6a-8213-1ceb08a23358 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:13.129877 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459a633-deea-4a6a-8213-1ceb08a23358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:13.151375 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-8459a633-deea-4a6a-8213-1ceb08a23358 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:13.152648 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8459a633-deea-4a6a-8213-1ceb08a23358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:13.263282 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1167/2335] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:13.275341 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db74487-7bb6-4914-843c-439edfc11c45 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:13.282227 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7db74487-7bb6-4914-843c-439edfc11c45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:13.463231 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db74487-7bb6-4914-843c-439edfc11c45 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:13.470716 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c7ae22-4910-4ee2-a56c-6cc38785f1e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:13.472084 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7ae22-4910-4ee2-a56c-6cc38785f1e0 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:13.472509 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7ae22-4910-4ee2-a56c-6cc38785f1e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:13.473225 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7ae22-4910-4ee2-a56c-6cc38785f1e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:13.490092 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c7ae22-4910-4ee2-a56c-6cc38785f1e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:13.492666 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c7ae22-4910-4ee2-a56c-6cc38785f1e0 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:13.517203 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1169/2336] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:04:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:04:14.019391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8459a633-deea-4a6a-8213-1ceb08a23358 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:14.021309 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8459a633-deea-4a6a-8213-1ceb08a23358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:14.040232 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8459a633-deea-4a6a-8213-1ceb08a23358 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:14.046215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac0924-5e87-4822-90e5-2a39fafecc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:14.048629 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac0924-5e87-4822-90e5-2a39fafecc2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:14.049218 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac0924-5e87-4822-90e5-2a39fafecc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:14.050239 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac0924-5e87-4822-90e5-2a39fafecc2f admin admin] PATH_INFO: `/v3/projects/d55914be74874d4fb313c88e0c804e08` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:14.053751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac0924-5e87-4822-90e5-2a39fafecc2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d55914be74874d4fb313c88e0c804e08)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:14.056008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac0924-5e87-4822-90e5-2a39fafecc2f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:14.131275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e6ac0924-5e87-4822-90e5-2a39fafecc2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd55914be74874d4fb313c88e0c804e08', 'request_id': 'req-e6ac0924-5e87-4822-90e5-2a39fafecc2f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:14.138314 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e6ac0924-5e87-4822-90e5-2a39fafecc2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd55914be74874d4fb313c88e0c804e08', 'request_id': 'req-e6ac0924-5e87-4822-90e5-2a39fafecc2f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:14.145898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e6ac0924-5e87-4822-90e5-2a39fafecc2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd55914be74874d4fb313c88e0c804e08', 'request_id': 'req-e6ac0924-5e87-4822-90e5-2a39fafecc2f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:14.165916 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e6ac0924-5e87-4822-90e5-2a39fafecc2f admin admin] The token cache is being invalidate because project d55914be74874d4fb313c88e0c804e08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:04:14.165916 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e6ac0924-5e87-4822-90e5-2a39fafecc2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:14.166842 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e6ac0924-5e87-4822-90e5-2a39fafecc2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:14.168529 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1168/2337] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:04:14 2022] DELETE /identity/v3/projects/d55914be74874d4fb313c88e0c804e08 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:04:14.174036 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a912c4c-5632-4f07-b285-fb8170025a48 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:14.174559 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a912c4c-5632-4f07-b285-fb8170025a48 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:14.174639 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a912c4c-5632-4f07-b285-fb8170025a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:14.198234 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-0a912c4c-5632-4f07-b285-fb8170025a48 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:14.199475 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0a912c4c-5632-4f07-b285-fb8170025a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:14.283542 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1170/2338] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:14.285891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af28b68-f562-4be6-8bb0-d49dd1dcb094 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:14.286216 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af28b68-f562-4be6-8bb0-d49dd1dcb094 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:14.286519 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af28b68-f562-4be6-8bb0-d49dd1dcb094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:14.309040 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-8af28b68-f562-4be6-8bb0-d49dd1dcb094 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:14.309956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8af28b68-f562-4be6-8bb0-d49dd1dcb094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:14.325175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1169/2339] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:04:14.327087 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a912c4c-5632-4f07-b285-fb8170025a48 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:14.341599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a912c4c-5632-4f07-b285-fb8170025a48 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:14.358782 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14759e1a-06df-4473-8aac-ad4349891cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:14.360222 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14759e1a-06df-4473-8aac-ad4349891cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:14.360989 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14759e1a-06df-4473-8aac-ad4349891cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:14.361274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14759e1a-06df-4473-8aac-ad4349891cf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:14.362017 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14759e1a-06df-4473-8aac-ad4349891cf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:04:14.362352 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14759e1a-06df-4473-8aac-ad4349891cf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:14.370777 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14759e1a-06df-4473-8aac-ad4349891cf2 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:14.397992 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1171/2340] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:04:14.400325 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af28b68-f562-4be6-8bb0-d49dd1dcb094 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:14.418985 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af28b68-f562-4be6-8bb0-d49dd1dcb094 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:14.439152 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811c9b9-a082-4539-b964-f48b3085b7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:14.440455 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811c9b9-a082-4539-b964-f48b3085b7e6 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:14.440892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811c9b9-a082-4539-b964-f48b3085b7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:14.441459 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811c9b9-a082-4539-b964-f48b3085b7e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:14.442948 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811c9b9-a082-4539-b964-f48b3085b7e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:14.445152 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811c9b9-a082-4539-b964-f48b3085b7e6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:14.488122 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1170/2341] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:04:14 2022] POST /identity/v3/projects => generated 363 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:04:14.488846 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14759e1a-06df-4473-8aac-ad4349891cf2 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:14.499716 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14759e1a-06df-4473-8aac-ad4349891cf2 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:14.572178 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96489e-2679-4ae4-8631-685c3faccc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:14.573164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96489e-2679-4ae4-8631-685c3faccc02 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:14.573601 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96489e-2679-4ae4-8631-685c3faccc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:14.574073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96489e-2679-4ae4-8631-685c3faccc02 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:14.574941 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a96489e-2679-4ae4-8631-685c3faccc02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:14.577442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a96489e-2679-4ae4-8631-685c3faccc02 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:14.604497 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1172/2342] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:04:14 2022] POST /identity/v3/users => generated 282 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:04:14.608287 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811c9b9-a082-4539-b964-f48b3085b7e6 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:14.632248 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811c9b9-a082-4539-b964-f48b3085b7e6 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:14.646489 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d3c84d-db1a-4b04-b8ed-69d47451f5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:14.647238 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3c84d-db1a-4b04-b8ed-69d47451f5f4 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:14.647480 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3c84d-db1a-4b04-b8ed-69d47451f5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:14.647809 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3c84d-db1a-4b04-b8ed-69d47451f5f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:14.649013 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d3c84d-db1a-4b04-b8ed-69d47451f5f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:14.650359 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d3c84d-db1a-4b04-b8ed-69d47451f5f4 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:14.664510 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1171/2343] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:04:14.668626 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96489e-2679-4ae4-8631-685c3faccc02 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:14.679955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96489e-2679-4ae4-8631-685c3faccc02 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:14.687404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c433cad-b1a5-42de-bc44-835e5b1ed915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:14.688283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c433cad-b1a5-42de-bc44-835e5b1ed915 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:14.689187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c433cad-b1a5-42de-bc44-835e5b1ed915 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:14.689484 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c433cad-b1a5-42de-bc44-835e5b1ed915 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:14.690220 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c433cad-b1a5-42de-bc44-835e5b1ed915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:14.692888 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c433cad-b1a5-42de-bc44-835e5b1ed915 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:14.706165 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1173/2344] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:04:14.719545 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d3c84d-db1a-4b04-b8ed-69d47451f5f4 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:14.725997 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c433cad-b1a5-42de-bc44-835e5b1ed915 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:14.728616 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4c433cad-b1a5-42de-bc44-835e5b1ed915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:14.746163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d3c84d-db1a-4b04-b8ed-69d47451f5f4 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:14.758294 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7786eed-7533-41d4-93ef-570f0122fdea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:14.759371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c433cad-b1a5-42de-bc44-835e5b1ed915 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:14.760407 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7786eed-7533-41d4-93ef-570f0122fdea admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:14.760686 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7786eed-7533-41d4-93ef-570f0122fdea admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:14.761937 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7786eed-7533-41d4-93ef-570f0122fdea admin admin] PATH_INFO: `/v3/projects/26203aeaad9e4efab1c9b1eab2d9e95e/users/05e964233f2b441f9b146c81a5c71471/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:14.771845 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167793d-2aa6-4aae-9bc5-c9bcfaea4fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:14.772995 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167793d-2aa6-4aae-9bc5-c9bcfaea4fea admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:14.775582 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167793d-2aa6-4aae-9bc5-c9bcfaea4fea admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:14.776128 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167793d-2aa6-4aae-9bc5-c9bcfaea4fea admin admin] PATH_INFO: `/v3/users/0f9a151d58eb4c8a8e99c1e35e6723f6` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:14.778616 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167793d-2aa6-4aae-9bc5-c9bcfaea4fea admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f9a151d58eb4c8a8e99c1e35e6723f6)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:14.780733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167793d-2aa6-4aae-9bc5-c9bcfaea4fea admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:14.792695 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7786eed-7533-41d4-93ef-570f0122fdea admin admin] RBAC: Authorizing `identity:create_grant(project_id=26203aeaad9e4efab1c9b1eab2d9e95e, user_id=05e964233f2b441f9b146c81a5c71471, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:14.804898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7786eed-7533-41d4-93ef-570f0122fdea admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:14.846001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1172/2345] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:04:14 2022] PUT /identity/v3/projects/26203aeaad9e4efab1c9b1eab2d9e95e/users/05e964233f2b441f9b146c81a5c71471/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:04:14.848205 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ca0d8-b5f6-4ea9-81e6-d9deb2f16419 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:14.848549 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ca0d8-b5f6-4ea9-81e6-d9deb2f16419 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:14.849761 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ca0d8-b5f6-4ea9-81e6-d9deb2f16419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:14.856833 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a167793d-2aa6-4aae-9bc5-c9bcfaea4fea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f9a151d58eb4c8a8e99c1e35e6723f6'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:14.868638 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a167793d-2aa6-4aae-9bc5-c9bcfaea4fea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f9a151d58eb4c8a8e99c1e35e6723f6'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:14.881577 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a167793d-2aa6-4aae-9bc5-c9bcfaea4fea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f9a151d58eb4c8a8e99c1e35e6723f6'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:14.882580 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a167793d-2aa6-4aae-9bc5-c9bcfaea4fea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f9a151d58eb4c8a8e99c1e35e6723f6'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:14.902575 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d11ca0d8-b5f6-4ea9-81e6-d9deb2f16419 None None] MFA Rules not processed for user `05e964233f2b441f9b146c81a5c71471`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:14.903865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d11ca0d8-b5f6-4ea9-81e6-d9deb2f16419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:14.910060 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a167793d-2aa6-4aae-9bc5-c9bcfaea4fea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f9a151d58eb4c8a8e99c1e35e6723f6'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:14.923859 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1174/2346] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:04:14 2022] DELETE /identity/v3/users/0f9a151d58eb4c8a8e99c1e35e6723f6 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:04:14.929467 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6391dce-f4e3-4c87-acc4-c33af65e94f9 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:14.929763 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6391dce-f4e3-4c87-acc4-c33af65e94f9 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:14.930045 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6391dce-f4e3-4c87-acc4-c33af65e94f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:14.964276 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-b6391dce-f4e3-4c87-acc4-c33af65e94f9 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:14.965737 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b6391dce-f4e3-4c87-acc4-c33af65e94f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:15.001842 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1173/2347] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:14 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:15.026945 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ca0d8-b5f6-4ea9-81e6-d9deb2f16419 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:15.028444 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d11ca0d8-b5f6-4ea9-81e6-d9deb2f16419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:15.064433 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1175/2348] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:15.070776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6391dce-f4e3-4c87-acc4-c33af65e94f9 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:15.073169 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b6391dce-f4e3-4c87-acc4-c33af65e94f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:15.097309 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ca0d8-b5f6-4ea9-81e6-d9deb2f16419 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:15.160491 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475b8f1-3125-4134-9acd-33d7d0fe9167 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:15.161932 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475b8f1-3125-4134-9acd-33d7d0fe9167 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:15.165909 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475b8f1-3125-4134-9acd-33d7d0fe9167 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:15.166245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475b8f1-3125-4134-9acd-33d7d0fe9167 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:15.181525 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6391dce-f4e3-4c87-acc4-c33af65e94f9 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:15.189077 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcfe416-0204-40cd-9030-ec77ec4d2882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:15.190531 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfe416-0204-40cd-9030-ec77ec4d2882 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:15.191050 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfe416-0204-40cd-9030-ec77ec4d2882 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:15.191578 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfe416-0204-40cd-9030-ec77ec4d2882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:15.203780 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcfe416-0204-40cd-9030-ec77ec4d2882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:15.205901 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcfe416-0204-40cd-9030-ec77ec4d2882 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:15.219361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1176/2349] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:04:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:04:15.240139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475b8f1-3125-4134-9acd-33d7d0fe9167 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:15.242232 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475b8f1-3125-4134-9acd-33d7d0fe9167 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:15.269812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1174/2350] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:04:15 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 23:04:15.700137 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcfe416-0204-40cd-9030-ec77ec4d2882 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:15.701626 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0fcfe416-0204-40cd-9030-ec77ec4d2882 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:15.722875 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcfe416-0204-40cd-9030-ec77ec4d2882 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:15.727944 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f3db2-391e-493a-9aa5-37d256437528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:15.728870 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f3db2-391e-493a-9aa5-37d256437528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:15.729218 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f3db2-391e-493a-9aa5-37d256437528 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:15.729620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f3db2-391e-493a-9aa5-37d256437528 admin admin] PATH_INFO: `/v3/projects/fcbf3360a1654ddbbc5812f5ea706eb6` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:15.731397 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834f3db2-391e-493a-9aa5-37d256437528 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcbf3360a1654ddbbc5812f5ea706eb6)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:15.735038 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834f3db2-391e-493a-9aa5-37d256437528 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:15.821258 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-834f3db2-391e-493a-9aa5-37d256437528 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcbf3360a1654ddbbc5812f5ea706eb6', 'request_id': 'req-834f3db2-391e-493a-9aa5-37d256437528'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:15.828609 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-834f3db2-391e-493a-9aa5-37d256437528 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcbf3360a1654ddbbc5812f5ea706eb6', 'request_id': 'req-834f3db2-391e-493a-9aa5-37d256437528'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:15.838331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-834f3db2-391e-493a-9aa5-37d256437528 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcbf3360a1654ddbbc5812f5ea706eb6', 'request_id': 'req-834f3db2-391e-493a-9aa5-37d256437528'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:15.853655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-834f3db2-391e-493a-9aa5-37d256437528 admin admin] The token cache is being invalidate because project fcbf3360a1654ddbbc5812f5ea706eb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:04:15.854095 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-834f3db2-391e-493a-9aa5-37d256437528 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:15.854842 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-834f3db2-391e-493a-9aa5-37d256437528 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:15.856592 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1177/2351] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:04:15 2022] DELETE /identity/v3/projects/fcbf3360a1654ddbbc5812f5ea706eb6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:04:16.337453 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b38344-486c-489c-94d7-5a7c85527821 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:16.338006 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b38344-486c-489c-94d7-5a7c85527821 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:16.338006 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b38344-486c-489c-94d7-5a7c85527821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:16.365417 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-c4b38344-486c-489c-94d7-5a7c85527821 None None] MFA Rules not processed for user `19fbe84526dd48a6be14c1aebf5713c0`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:16.366718 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c4b38344-486c-489c-94d7-5a7c85527821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:16.432311 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1175/2352] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:04:16.436719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f3db2-391e-493a-9aa5-37d256437528 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:16.438398 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-834f3db2-391e-493a-9aa5-37d256437528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:16.508697 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f3db2-391e-493a-9aa5-37d256437528 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:16.513770 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974ff34-0fdb-4a06-a493-03b88dc62c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:16.515000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974ff34-0fdb-4a06-a493-03b88dc62c42 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:16.515274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974ff34-0fdb-4a06-a493-03b88dc62c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:16.515540 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974ff34-0fdb-4a06-a493-03b88dc62c42 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:16.516366 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974ff34-0fdb-4a06-a493-03b88dc62c42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:16.518293 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974ff34-0fdb-4a06-a493-03b88dc62c42 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:16.551715 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1178/2353] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:04:16 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:04:16.555642 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b38344-486c-489c-94d7-5a7c85527821 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:16.569384 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b38344-486c-489c-94d7-5a7c85527821 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:16.629423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf1d5d-bacc-41e7-bfd9-71f4f269311a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:16.630226 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf1d5d-bacc-41e7-bfd9-71f4f269311a admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:16.630575 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf1d5d-bacc-41e7-bfd9-71f4f269311a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:16.631061 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf1d5d-bacc-41e7-bfd9-71f4f269311a admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:16.632025 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcf1d5d-bacc-41e7-bfd9-71f4f269311a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:16.634042 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcf1d5d-bacc-41e7-bfd9-71f4f269311a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:16.660957 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1176/2354] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:04:16 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:04:16.662196 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974ff34-0fdb-4a06-a493-03b88dc62c42 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:16.675642 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974ff34-0fdb-4a06-a493-03b88dc62c42 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:16.683209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcfd14-8f60-4cb8-89bc-74d4cbb0a5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:16.684197 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcfd14-8f60-4cb8-89bc-74d4cbb0a5d5 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:16.684401 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcfd14-8f60-4cb8-89bc-74d4cbb0a5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:16.684694 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcfd14-8f60-4cb8-89bc-74d4cbb0a5d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:16.685534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcfd14-8f60-4cb8-89bc-74d4cbb0a5d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:16.687272 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcfd14-8f60-4cb8-89bc-74d4cbb0a5d5 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:16.702685 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1179/2355] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:04:16.705062 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf1d5d-bacc-41e7-bfd9-71f4f269311a admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:16.715723 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf1d5d-bacc-41e7-bfd9-71f4f269311a admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:16.723903 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f0b2df-676a-442e-a127-45c8ba64fe8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:16.724999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0b2df-676a-442e-a127-45c8ba64fe8e admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:16.725280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0b2df-676a-442e-a127-45c8ba64fe8e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:16.725669 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0b2df-676a-442e-a127-45c8ba64fe8e admin admin] PATH_INFO: `/v3/projects/3d9e04b82a614f86be6d631e4bdc6918/users/ea3b7736ecf041a487b9e983f3fc4e81/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:16.742033 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f0b2df-676a-442e-a127-45c8ba64fe8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d9e04b82a614f86be6d631e4bdc6918, user_id=ea3b7736ecf041a487b9e983f3fc4e81, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:16.744842 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f0b2df-676a-442e-a127-45c8ba64fe8e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:16.762249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1177/2356] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:04:16 2022] PUT /identity/v3/projects/3d9e04b82a614f86be6d631e4bdc6918/users/ea3b7736ecf041a487b9e983f3fc4e81/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:04:16.766805 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcfd14-8f60-4cb8-89bc-74d4cbb0a5d5 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:16.782733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcfd14-8f60-4cb8-89bc-74d4cbb0a5d5 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:16.848481 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185ea72-093a-4ee3-bf95-6a2bd61e499e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:16.849157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185ea72-093a-4ee3-bf95-6a2bd61e499e admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:16.849440 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185ea72-093a-4ee3-bf95-6a2bd61e499e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:16.849719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185ea72-093a-4ee3-bf95-6a2bd61e499e admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:16.850478 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185ea72-093a-4ee3-bf95-6a2bd61e499e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:16.852109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185ea72-093a-4ee3-bf95-6a2bd61e499e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:16.863843 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1180/2357] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:04:16.867855 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f0b2df-676a-442e-a127-45c8ba64fe8e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:16.878870 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f0b2df-676a-442e-a127-45c8ba64fe8e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:16.883803 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51fc966-5496-478c-ac3d-719cb5af03c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:16.884675 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fc966-5496-478c-ac3d-719cb5af03c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:16.884954 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fc966-5496-478c-ac3d-719cb5af03c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:16.885225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fc966-5496-478c-ac3d-719cb5af03c8 admin admin] PATH_INFO: `/v3/projects/3d9e04b82a614f86be6d631e4bdc6918/users/ea3b7736ecf041a487b9e983f3fc4e81/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:16.887192 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51fc966-5496-478c-ac3d-719cb5af03c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d9e04b82a614f86be6d631e4bdc6918, user_id=ea3b7736ecf041a487b9e983f3fc4e81, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:16.890520 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51fc966-5496-478c-ac3d-719cb5af03c8 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:16.900590 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1178/2358] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:04:16 2022] PUT /identity/v3/projects/3d9e04b82a614f86be6d631e4bdc6918/users/ea3b7736ecf041a487b9e983f3fc4e81/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:04:20.385246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185ea72-093a-4ee3-bf95-6a2bd61e499e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:20.387590 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-8185ea72-093a-4ee3-bf95-6a2bd61e499e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:20.411418 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185ea72-093a-4ee3-bf95-6a2bd61e499e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:20.570088 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94199375-ced3-40b6-adb5-86a43c5614d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:20.571370 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94199375-ced3-40b6-adb5-86a43c5614d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:20.571874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94199375-ced3-40b6-adb5-86a43c5614d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:20.572372 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94199375-ced3-40b6-adb5-86a43c5614d5 admin admin] PATH_INFO: `/v3/users/4546c9ee47ce475bbf2aaad9794dcfa1` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:20.575170 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94199375-ced3-40b6-adb5-86a43c5614d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4546c9ee47ce475bbf2aaad9794dcfa1)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:20.577709 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94199375-ced3-40b6-adb5-86a43c5614d5 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:20.661658 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-94199375-ced3-40b6-adb5-86a43c5614d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4546c9ee47ce475bbf2aaad9794dcfa1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:20.669903 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-94199375-ced3-40b6-adb5-86a43c5614d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4546c9ee47ce475bbf2aaad9794dcfa1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:20.687333 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-94199375-ced3-40b6-adb5-86a43c5614d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4546c9ee47ce475bbf2aaad9794dcfa1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:20.688746 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-94199375-ced3-40b6-adb5-86a43c5614d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4546c9ee47ce475bbf2aaad9794dcfa1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:20.707688 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-94199375-ced3-40b6-adb5-86a43c5614d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4546c9ee47ce475bbf2aaad9794dcfa1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:20.716741 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1181/2359] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:04:20 2022] DELETE /identity/v3/users/4546c9ee47ce475bbf2aaad9794dcfa1 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:04:20.722038 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf3d1c-2cc1-463a-a74a-93734ae13056 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:20.722594 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf3d1c-2cc1-463a-a74a-93734ae13056 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:20.722594 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf3d1c-2cc1-463a-a74a-93734ae13056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:20.763968 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-62bf3d1c-2cc1-463a-a74a-93734ae13056 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:20.767125 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-62bf3d1c-2cc1-463a-a74a-93734ae13056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:20.872487 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1179/2360] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:20.879560 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94199375-ced3-40b6-adb5-86a43c5614d5 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:20.884109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-94199375-ced3-40b6-adb5-86a43c5614d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:20.984058 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94199375-ced3-40b6-adb5-86a43c5614d5 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:20.991981 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285bfbf-65de-4286-a2fc-eb6eb988a3b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:20.992618 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285bfbf-65de-4286-a2fc-eb6eb988a3b5 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:20.992901 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285bfbf-65de-4286-a2fc-eb6eb988a3b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:20.993176 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285bfbf-65de-4286-a2fc-eb6eb988a3b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:21.019621 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2285bfbf-65de-4286-a2fc-eb6eb988a3b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:21.021261 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2285bfbf-65de-4286-a2fc-eb6eb988a3b5 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:21.042508 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1182/2361] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:04:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:04:21.667158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf3d1c-2cc1-463a-a74a-93734ae13056 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:21.672567 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-62bf3d1c-2cc1-463a-a74a-93734ae13056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:21.712461 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf3d1c-2cc1-463a-a74a-93734ae13056 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:21.718556 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d9fa3-e476-43ea-b096-9c0c58551417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:21.719807 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d9fa3-e476-43ea-b096-9c0c58551417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:21.720319 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d9fa3-e476-43ea-b096-9c0c58551417 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:21.720800 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d9fa3-e476-43ea-b096-9c0c58551417 admin admin] PATH_INFO: `/v3/projects/cf71a634012b41eb9e7540df3b1dddb3` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:21.723066 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d9fa3-e476-43ea-b096-9c0c58551417 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf71a634012b41eb9e7540df3b1dddb3)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:21.726332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d9fa3-e476-43ea-b096-9c0c58551417 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:21.812636 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-785d9fa3-e476-43ea-b096-9c0c58551417 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf71a634012b41eb9e7540df3b1dddb3', 'request_id': 'req-785d9fa3-e476-43ea-b096-9c0c58551417'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:21.826219 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-785d9fa3-e476-43ea-b096-9c0c58551417 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf71a634012b41eb9e7540df3b1dddb3', 'request_id': 'req-785d9fa3-e476-43ea-b096-9c0c58551417'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:21.834206 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-785d9fa3-e476-43ea-b096-9c0c58551417 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf71a634012b41eb9e7540df3b1dddb3', 'request_id': 'req-785d9fa3-e476-43ea-b096-9c0c58551417'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:21.859684 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-785d9fa3-e476-43ea-b096-9c0c58551417 admin admin] The token cache is being invalidate because project cf71a634012b41eb9e7540df3b1dddb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:04:21.861878 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-785d9fa3-e476-43ea-b096-9c0c58551417 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:21.866059 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-785d9fa3-e476-43ea-b096-9c0c58551417 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:21.870606 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1180/2362] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:04:21 2022] DELETE /identity/v3/projects/cf71a634012b41eb9e7540df3b1dddb3 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:04:23.147508 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9587e-8a51-49fa-8aed-5f0c6f5293f1 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:23.148045 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9587e-8a51-49fa-8aed-5f0c6f5293f1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:23.148045 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9587e-8a51-49fa-8aed-5f0c6f5293f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:23.184022 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-52e9587e-8a51-49fa-8aed-5f0c6f5293f1 None None] MFA Rules not processed for user `ea3b7736ecf041a487b9e983f3fc4e81`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:23.185240 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-52e9587e-8a51-49fa-8aed-5f0c6f5293f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:23.297412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1183/2363] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:23.311393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d9fa3-e476-43ea-b096-9c0c58551417 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:23.313095 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-785d9fa3-e476-43ea-b096-9c0c58551417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:23.413179 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d9fa3-e476-43ea-b096-9c0c58551417 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:23.417638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c52d02-13fb-4a54-8e69-eba99311a0dc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81caf0558f2449cbbc8f4cd9446f363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:23.418574 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c52d02-13fb-4a54-8e69-eba99311a0dc service glance] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:23.418822 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c52d02-13fb-4a54-8e69-eba99311a0dc service glance] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:23.419084 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c52d02-13fb-4a54-8e69-eba99311a0dc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:23.421620 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d5c52d02-13fb-4a54-8e69-eba99311a0dc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:23.497187 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c52d02-13fb-4a54-8e69-eba99311a0dc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:23.499012 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c52d02-13fb-4a54-8e69-eba99311a0dc service glance] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:23.510938 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1181/2364] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:04:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:04:23.645530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc03dd7-55ca-4a8a-8192-a9626c2522d4 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:23.645912 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc03dd7-55ca-4a8a-8192-a9626c2522d4 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:23.646302 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc03dd7-55ca-4a8a-8192-a9626c2522d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:23.666703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-acc03dd7-55ca-4a8a-8192-a9626c2522d4 None None] MFA Rules not processed for user `1a6f50dcebf146baae8a9f0b5b112795`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:23.667941 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-acc03dd7-55ca-4a8a-8192-a9626c2522d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:23.744785 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1184/2365] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 9 23:04:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:23.759483 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c52d02-13fb-4a54-8e69-eba99311a0dc service glance] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:23.760954 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d5c52d02-13fb-4a54-8e69-eba99311a0dc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:23.866529 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c52d02-13fb-4a54-8e69-eba99311a0dc service glance] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:23.872926 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f60c545-a35e-4a2e-a569-7434d227146b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cb6c294ce1842b68a6bc73ac381ed6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:23.874276 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60c545-a35e-4a2e-a569-7434d227146b service swift] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:23.874878 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60c545-a35e-4a2e-a569-7434d227146b service swift] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:23.875262 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60c545-a35e-4a2e-a569-7434d227146b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:23.886083 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f60c545-a35e-4a2e-a569-7434d227146b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:23.887660 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f60c545-a35e-4a2e-a569-7434d227146b service swift] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:23.902274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1182/2366] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 9 23:04:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 23:04:31.523396 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37599d-5de5-437e-b9e0-14ad8ba4ba4c None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:31.529179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37599d-5de5-437e-b9e0-14ad8ba4ba4c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:31.529841 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37599d-5de5-437e-b9e0-14ad8ba4ba4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:31.576258 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-bf37599d-5de5-437e-b9e0-14ad8ba4ba4c None None] MFA Rules not processed for user `84a45506c19c4cda8173c164a097f925`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:31.577846 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-bf37599d-5de5-437e-b9e0-14ad8ba4ba4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:31.693415 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1185/2367] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:33.538643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9022796-1413-410d-9b07-a70be143cf9e None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:33.540050 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9022796-1413-410d-9b07-a70be143cf9e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:33.540347 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9022796-1413-410d-9b07-a70be143cf9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:33.564617 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-c9022796-1413-410d-9b07-a70be143cf9e None None] MFA Rules not processed for user `e41b4feaa9be44e89210b70c19ef10ab`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:33.566106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c9022796-1413-410d-9b07-a70be143cf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:33.647785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1183/2368] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:33.956477 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fba3a5-e319-4ff4-8732-e2ade03d27ad None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:33.957076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fba3a5-e319-4ff4-8732-e2ade03d27ad None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:33.957175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fba3a5-e319-4ff4-8732-e2ade03d27ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:33.975024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-57fba3a5-e319-4ff4-8732-e2ade03d27ad None None] MFA Rules not processed for user `19fbe84526dd48a6be14c1aebf5713c0`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:33.976221 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-57fba3a5-e319-4ff4-8732-e2ade03d27ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:33.983874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1186/2369] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:36.489568 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06659d-4681-4dc7-8be8-e58079a94373 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:36.490613 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06659d-4681-4dc7-8be8-e58079a94373 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:36.491106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06659d-4681-4dc7-8be8-e58079a94373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:36.512122 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-3b06659d-4681-4dc7-8be8-e58079a94373 None None] MFA Rules not processed for user `1a6f50dcebf146baae8a9f0b5b112795`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:36.513523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3b06659d-4681-4dc7-8be8-e58079a94373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:36.546217 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1184/2370] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 9 23:04:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:36.571405 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fba3a5-e319-4ff4-8732-e2ade03d27ad None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:36.620622 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fba3a5-e319-4ff4-8732-e2ade03d27ad None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:36.634779 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d7fac-6aa0-4f85-9ff9-7063885f9ac6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cb6c294ce1842b68a6bc73ac381ed6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:36.640960 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d7fac-6aa0-4f85-9ff9-7063885f9ac6 service swift] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:36.641477 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d7fac-6aa0-4f85-9ff9-7063885f9ac6 service swift] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:36.642449 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d7fac-6aa0-4f85-9ff9-7063885f9ac6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:36.654572 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d7fac-6aa0-4f85-9ff9-7063885f9ac6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:36.656483 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d7fac-6aa0-4f85-9ff9-7063885f9ac6 service swift] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:36.666826 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1187/2371] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 9 23:04:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 23:04:55.261523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06659d-4681-4dc7-8be8-e58079a94373 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:55.264480 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3b06659d-4681-4dc7-8be8-e58079a94373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:55.420307 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06659d-4681-4dc7-8be8-e58079a94373 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:55.425905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ce694-62fb-48ab-84ff-909c262332af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:55.428152 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ce694-62fb-48ab-84ff-909c262332af admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:55.428434 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ce694-62fb-48ab-84ff-909c262332af admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:55.428703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ce694-62fb-48ab-84ff-909c262332af admin admin] PATH_INFO: `/v3/users/05e964233f2b441f9b146c81a5c71471` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:55.430182 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ce694-62fb-48ab-84ff-909c262332af admin admin] RBAC: Authorizing `identity:delete_user(user_id=05e964233f2b441f9b146c81a5c71471)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:55.432314 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ce694-62fb-48ab-84ff-909c262332af admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:55.488858 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-353ce694-62fb-48ab-84ff-909c262332af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05e964233f2b441f9b146c81a5c71471'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:55.495893 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-353ce694-62fb-48ab-84ff-909c262332af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05e964233f2b441f9b146c81a5c71471'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:55.496651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-353ce694-62fb-48ab-84ff-909c262332af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05e964233f2b441f9b146c81a5c71471'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:55.506325 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-353ce694-62fb-48ab-84ff-909c262332af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05e964233f2b441f9b146c81a5c71471'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:55.507021 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-353ce694-62fb-48ab-84ff-909c262332af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05e964233f2b441f9b146c81a5c71471'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:55.525167 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1185/2372] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:04:55 2022] DELETE /identity/v3/users/05e964233f2b441f9b146c81a5c71471 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:04:55.530308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935e2bb-3db6-416f-916e-a415d9bd66c9 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:55.530840 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935e2bb-3db6-416f-916e-a415d9bd66c9 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:55.531039 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935e2bb-3db6-416f-916e-a415d9bd66c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:55.558503 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5935e2bb-3db6-416f-916e-a415d9bd66c9 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:55.559729 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5935e2bb-3db6-416f-916e-a415d9bd66c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:55.630244 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1188/2373] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:55.646817 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ce694-62fb-48ab-84ff-909c262332af admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:55.648127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-353ce694-62fb-48ab-84ff-909c262332af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:55.751115 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ce694-62fb-48ab-84ff-909c262332af admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:55.755300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883574b-0bca-40c9-af7d-486935e69b82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:55.756181 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883574b-0bca-40c9-af7d-486935e69b82 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:55.756463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883574b-0bca-40c9-af7d-486935e69b82 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:55.756735 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883574b-0bca-40c9-af7d-486935e69b82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:55.767620 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d883574b-0bca-40c9-af7d-486935e69b82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:55.769178 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d883574b-0bca-40c9-af7d-486935e69b82 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:55.780278 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1186/2374] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:04:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:04:56.069254 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935e2bb-3db6-416f-916e-a415d9bd66c9 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:56.070786 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5935e2bb-3db6-416f-916e-a415d9bd66c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:56.085199 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935e2bb-3db6-416f-916e-a415d9bd66c9 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:56.092994 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018fe2f-ea98-46d5-ab61-25347677950b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:56.093860 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018fe2f-ea98-46d5-ab61-25347677950b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:56.094178 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018fe2f-ea98-46d5-ab61-25347677950b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:56.094475 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018fe2f-ea98-46d5-ab61-25347677950b admin admin] PATH_INFO: `/v3/projects/26203aeaad9e4efab1c9b1eab2d9e95e` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:56.095691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018fe2f-ea98-46d5-ab61-25347677950b admin admin] RBAC: Authorizing `identity:delete_project(project_id=26203aeaad9e4efab1c9b1eab2d9e95e)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:56.098690 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018fe2f-ea98-46d5-ab61-25347677950b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:56.169641 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c018fe2f-ea98-46d5-ab61-25347677950b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26203aeaad9e4efab1c9b1eab2d9e95e', 'request_id': 'req-c018fe2f-ea98-46d5-ab61-25347677950b'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:56.177079 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c018fe2f-ea98-46d5-ab61-25347677950b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26203aeaad9e4efab1c9b1eab2d9e95e', 'request_id': 'req-c018fe2f-ea98-46d5-ab61-25347677950b'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:56.183594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c018fe2f-ea98-46d5-ab61-25347677950b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26203aeaad9e4efab1c9b1eab2d9e95e', 'request_id': 'req-c018fe2f-ea98-46d5-ab61-25347677950b'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:56.202425 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c018fe2f-ea98-46d5-ab61-25347677950b admin admin] The token cache is being invalidate because project 26203aeaad9e4efab1c9b1eab2d9e95e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:04:56.202720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c018fe2f-ea98-46d5-ab61-25347677950b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:56.203474 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c018fe2f-ea98-46d5-ab61-25347677950b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:04:56.205023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1189/2375] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:04:56 2022] DELETE /identity/v3/projects/26203aeaad9e4efab1c9b1eab2d9e95e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:04:56.210104 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29a684-b82d-4394-9346-c11584f22672 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:56.210441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29a684-b82d-4394-9346-c11584f22672 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:56.210711 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29a684-b82d-4394-9346-c11584f22672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:56.257924 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-1a29a684-b82d-4394-9346-c11584f22672 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:56.259146 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1a29a684-b82d-4394-9346-c11584f22672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:56.317422 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1187/2376] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:56.326030 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7a0af-f92c-44cc-bcbe-f0d61bd7ebcc None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:56.326862 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7a0af-f92c-44cc-bcbe-f0d61bd7ebcc None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:56.327143 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7a0af-f92c-44cc-bcbe-f0d61bd7ebcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:56.345436 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-7dc7a0af-f92c-44cc-bcbe-f0d61bd7ebcc None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:56.346680 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7dc7a0af-f92c-44cc-bcbe-f0d61bd7ebcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:56.354432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1190/2377] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:56.358679 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29a684-b82d-4394-9346-c11584f22672 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:56.368516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29a684-b82d-4394-9346-c11584f22672 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:56.373608 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719dfa4-4704-4e28-afe1-edcf6d37e826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:56.374294 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719dfa4-4704-4e28-afe1-edcf6d37e826 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:56.374568 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719dfa4-4704-4e28-afe1-edcf6d37e826 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:56.374834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719dfa4-4704-4e28-afe1-edcf6d37e826 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:56.375540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a719dfa4-4704-4e28-afe1-edcf6d37e826 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:04:56.375834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a719dfa4-4704-4e28-afe1-edcf6d37e826 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:56.377431 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a719dfa4-4704-4e28-afe1-edcf6d37e826 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:56.388387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1188/2378] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:04:56.393110 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7a0af-f92c-44cc-bcbe-f0d61bd7ebcc None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:56.402222 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7a0af-f92c-44cc-bcbe-f0d61bd7ebcc None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:56.407519 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed04513-0858-4006-8745-95c6440c77b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:56.408158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed04513-0858-4006-8745-95c6440c77b1 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:56.408441 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed04513-0858-4006-8745-95c6440c77b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:56.408717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed04513-0858-4006-8745-95c6440c77b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:56.409406 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed04513-0858-4006-8745-95c6440c77b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:56.411336 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed04513-0858-4006-8745-95c6440c77b1 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:56.441974 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1191/2379] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:04:56 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:04:56.445662 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719dfa4-4704-4e28-afe1-edcf6d37e826 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:56.456224 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719dfa4-4704-4e28-afe1-edcf6d37e826 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:56.521695 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae069d91-fd0b-48b6-ae94-3f78c037b266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:56.522393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae069d91-fd0b-48b6-ae94-3f78c037b266 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:56.526534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae069d91-fd0b-48b6-ae94-3f78c037b266 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:56.526919 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae069d91-fd0b-48b6-ae94-3f78c037b266 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:56.530544 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae069d91-fd0b-48b6-ae94-3f78c037b266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:56.532493 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae069d91-fd0b-48b6-ae94-3f78c037b266 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:56.554624 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1189/2380] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:04:56 2022] POST /identity/v3/users => generated 291 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:04:56.558648 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed04513-0858-4006-8745-95c6440c77b1 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:56.569948 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed04513-0858-4006-8745-95c6440c77b1 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:56.579225 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342aec0-96c7-4176-85dd-85c22b30d10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:56.579897 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342aec0-96c7-4176-85dd-85c22b30d10d admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:56.580179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342aec0-96c7-4176-85dd-85c22b30d10d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:56.580454 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342aec0-96c7-4176-85dd-85c22b30d10d admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:56.581178 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5342aec0-96c7-4176-85dd-85c22b30d10d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:56.582876 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5342aec0-96c7-4176-85dd-85c22b30d10d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:56.599490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1192/2381] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:04:56.601420 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae069d91-fd0b-48b6-ae94-3f78c037b266 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:56.612346 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae069d91-fd0b-48b6-ae94-3f78c037b266 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:56.616933 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f90392-3a81-41de-92be-8944feeb2b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:56.617584 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f90392-3a81-41de-92be-8944feeb2b18 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:56.617860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f90392-3a81-41de-92be-8944feeb2b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:56.618154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f90392-3a81-41de-92be-8944feeb2b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:56.618900 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f90392-3a81-41de-92be-8944feeb2b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:56.620581 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f90392-3a81-41de-92be-8944feeb2b18 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:56.631159 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1190/2382] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:04:56.635426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342aec0-96c7-4176-85dd-85c22b30d10d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:56.645920 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342aec0-96c7-4176-85dd-85c22b30d10d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:56.650555 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa37e37-bf49-4200-8ad5-fcdd9c2670e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:56.651433 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa37e37-bf49-4200-8ad5-fcdd9c2670e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:56.651713 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa37e37-bf49-4200-8ad5-fcdd9c2670e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:56.651986 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa37e37-bf49-4200-8ad5-fcdd9c2670e6 admin admin] PATH_INFO: `/v3/projects/9a54785a57d74beb87b5a31f928583e7/users/7d3e164b111b4c3784b220ef483bb5b9/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:56.666517 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa37e37-bf49-4200-8ad5-fcdd9c2670e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a54785a57d74beb87b5a31f928583e7, user_id=7d3e164b111b4c3784b220ef483bb5b9, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:56.669315 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa37e37-bf49-4200-8ad5-fcdd9c2670e6 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:56.683861 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1193/2383] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:04:56 2022] PUT /identity/v3/projects/9a54785a57d74beb87b5a31f928583e7/users/7d3e164b111b4c3784b220ef483bb5b9/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:04:56.689772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2c734-ad99-4e55-8d9b-01011a79421c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:56.690072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2c734-ad99-4e55-8d9b-01011a79421c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:56.690409 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2c734-ad99-4e55-8d9b-01011a79421c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:56.708738 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-3ec2c734-ad99-4e55-8d9b-01011a79421c None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:56.709936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3ec2c734-ad99-4e55-8d9b-01011a79421c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:56.767855 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1191/2384] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:56.777563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa37e37-bf49-4200-8ad5-fcdd9c2670e6 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:56.779366 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ffa37e37-bf49-4200-8ad5-fcdd9c2670e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:56.836919 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa37e37-bf49-4200-8ad5-fcdd9c2670e6 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:56.840558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9176dd-38cf-4ac2-8922-871eae89874b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:56.841205 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9176dd-38cf-4ac2-8922-871eae89874b service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:56.841486 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9176dd-38cf-4ac2-8922-871eae89874b service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:56.842733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9176dd-38cf-4ac2-8922-871eae89874b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:56.851833 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9176dd-38cf-4ac2-8922-871eae89874b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:56.853252 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9176dd-38cf-4ac2-8922-871eae89874b service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:56.862944 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1194/2385] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:04:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:04:58.161638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7189818-0adb-4880-a573-6b28bb2d729f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:58.162274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7189818-0adb-4880-a573-6b28bb2d729f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:58.162274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7189818-0adb-4880-a573-6b28bb2d729f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:58.192480 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e7189818-0adb-4880-a573-6b28bb2d729f None None] MFA Rules not processed for user `7d3e164b111b4c3784b220ef483bb5b9`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:58.194835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e7189818-0adb-4880-a573-6b28bb2d729f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:58.283956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1192/2386] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:04:58.298790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9176dd-38cf-4ac2-8922-871eae89874b service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:04:58.300276 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-aa9176dd-38cf-4ac2-8922-871eae89874b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:58.391912 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9176dd-38cf-4ac2-8922-871eae89874b service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:04:58.396273 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d771d47-6af1-469c-903b-f78de059b2e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:04:58.396916 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d771d47-6af1-469c-903b-f78de059b2e1 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:58.397196 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d771d47-6af1-469c-903b-f78de059b2e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:58.397568 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d771d47-6af1-469c-903b-f78de059b2e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:58.409116 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d771d47-6af1-469c-903b-f78de059b2e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:04:58.410618 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d771d47-6af1-469c-903b-f78de059b2e1 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:04:58.421776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1195/2387] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:04:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:04:59.871589 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75214c3-88b1-46cd-add0-3aad74b69097 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:04:59.872161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75214c3-88b1-46cd-add0-3aad74b69097 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:04:59.872161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75214c3-88b1-46cd-add0-3aad74b69097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:04:59.905763 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e75214c3-88b1-46cd-add0-3aad74b69097 None None] MFA Rules not processed for user `7d3e164b111b4c3784b220ef483bb5b9`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:04:59.907307 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e75214c3-88b1-46cd-add0-3aad74b69097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:04:59.914955 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1193/2388] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:04:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:05:13.281985 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d771d47-6af1-469c-903b-f78de059b2e1 service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:05:13.291093 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d771d47-6af1-469c-903b-f78de059b2e1 service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:05:13.294797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63249ded-52ba-4c8a-b199-4cf975f45baf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:05:13.296023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63249ded-52ba-4c8a-b199-4cf975f45baf service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:05:13.296302 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63249ded-52ba-4c8a-b199-4cf975f45baf service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:05:13.296577 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63249ded-52ba-4c8a-b199-4cf975f45baf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:05:13.298344 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-63249ded-52ba-4c8a-b199-4cf975f45baf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:05:13.360378 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63249ded-52ba-4c8a-b199-4cf975f45baf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:05:13.361816 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63249ded-52ba-4c8a-b199-4cf975f45baf service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:05:13.371432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1196/2389] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:05:13 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:05:15.949859 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd30bf-0f9a-4ed4-8544-00512fc026ed None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:05:15.951326 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd30bf-0f9a-4ed4-8544-00512fc026ed None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:05:15.951956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd30bf-0f9a-4ed4-8544-00512fc026ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:05:16.025299 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-6ffd30bf-0f9a-4ed4-8544-00512fc026ed None None] MFA Rules not processed for user `1a6f50dcebf146baae8a9f0b5b112795`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:05:16.026857 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-6ffd30bf-0f9a-4ed4-8544-00512fc026ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:05:16.158957 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1194/2390] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 9 23:05:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:05:16.185852 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63249ded-52ba-4c8a-b199-4cf975f45baf service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:05:16.188113 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-63249ded-52ba-4c8a-b199-4cf975f45baf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:05:16.328985 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63249ded-52ba-4c8a-b199-4cf975f45baf service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:05:16.353204 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe2884-f97e-458d-8f60-6c45f297d59e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cb6c294ce1842b68a6bc73ac381ed6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:05:16.355433 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe2884-f97e-458d-8f60-6c45f297d59e service swift] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:05:16.355433 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe2884-f97e-458d-8f60-6c45f297d59e service swift] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:05:16.355433 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe2884-f97e-458d-8f60-6c45f297d59e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:05:16.377912 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe2884-f97e-458d-8f60-6c45f297d59e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:05:16.379666 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe2884-f97e-458d-8f60-6c45f297d59e service swift] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:05:16.394029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1197/2391] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 9 23:05:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 23:05:18.066592 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd30bf-0f9a-4ed4-8544-00512fc026ed None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:05:18.069324 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-6ffd30bf-0f9a-4ed4-8544-00512fc026ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:05:18.186684 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd30bf-0f9a-4ed4-8544-00512fc026ed None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:05:18.206470 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d914f-ad8d-49f8-8343-4d3de91a1a2f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf9bc2fa959d4706b9f9e87509829059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:05:18.207059 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d914f-ad8d-49f8-8343-4d3de91a1a2f service placement] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:05:18.207337 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d914f-ad8d-49f8-8343-4d3de91a1a2f service placement] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:05:18.207608 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d914f-ad8d-49f8-8343-4d3de91a1a2f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:05:18.211220 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-505d914f-ad8d-49f8-8343-4d3de91a1a2f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:05:18.226097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505d914f-ad8d-49f8-8343-4d3de91a1a2f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:05:18.228047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505d914f-ad8d-49f8-8343-4d3de91a1a2f service placement] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:05:18.245858 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1195/2392] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:05:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:05:30.263061 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba9e6f-a8e9-48bb-8045-888db0d8478f None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:05:30.263612 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba9e6f-a8e9-48bb-8045-888db0d8478f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:05:30.263716 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba9e6f-a8e9-48bb-8045-888db0d8478f None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:05:30.265147 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1198/2393] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 9 23:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 23:05:30.271148 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c1fd2-6ba5-45b1-8b36-4be3ab864e9d None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:05:30.271648 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c1fd2-6ba5-45b1-8b36-4be3ab864e9d None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:05:30.271752 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c1fd2-6ba5-45b1-8b36-4be3ab864e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:05:30.321661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d36c1fd2-6ba5-45b1-8b36-4be3ab864e9d None None] MFA Rules not processed for user `02fcd7cc86134ce488c7dd75dfb174b8`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:05:30.322911 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d36c1fd2-6ba5-45b1-8b36-4be3ab864e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:05:30.401861 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1196/2394] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 9 23:05:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:05:30.414063 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba9e6f-a8e9-48bb-8045-888db0d8478f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:05:30.415790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-bcba9e6f-a8e9-48bb-8045-888db0d8478f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:05:30.493173 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba9e6f-a8e9-48bb-8045-888db0d8478f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:05:30.496622 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2641b70-8244-4a41-9943-834589fe2765 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81caf0558f2449cbbc8f4cd9446f363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:05:30.497251 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2641b70-8244-4a41-9943-834589fe2765 service glance] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:05:30.497529 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2641b70-8244-4a41-9943-834589fe2765 service glance] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:05:30.497823 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2641b70-8244-4a41-9943-834589fe2765 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:05:30.507632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2641b70-8244-4a41-9943-834589fe2765 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:05:30.509087 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2641b70-8244-4a41-9943-834589fe2765 service glance] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:05:30.519420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1199/2395] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:05:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:05:30.585779 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9468f4-8298-42f5-8e0c-e99fe9259f6e None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:05:30.586065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9468f4-8298-42f5-8e0c-e99fe9259f6e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:05:30.586387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9468f4-8298-42f5-8e0c-e99fe9259f6e None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:05:30.587750 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1197/2396] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 9 23:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 23:05:30.593382 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d4279-0a99-4ea3-b22f-309c3354e588 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:05:30.593675 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d4279-0a99-4ea3-b22f-309c3354e588 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:05:30.593954 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d4279-0a99-4ea3-b22f-309c3354e588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:05:30.612157 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e91d4279-0a99-4ea3-b22f-309c3354e588 None None] MFA Rules not processed for user `96fd3f6dd9e74ba7a4ad159e43e8ae96`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:05:30.613342 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e91d4279-0a99-4ea3-b22f-309c3354e588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:05:30.619783 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1200/2397] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 9 23:05:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:05:30.631326 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9468f4-8298-42f5-8e0c-e99fe9259f6e None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:05:30.632976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-6e9468f4-8298-42f5-8e0c-e99fe9259f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:05:30.646416 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9468f4-8298-42f5-8e0c-e99fe9259f6e None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:05:30.649882 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a790a87-7d15-404a-ac53-26f25100d2db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:05:30.650538 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a790a87-7d15-404a-ac53-26f25100d2db service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:05:30.650818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a790a87-7d15-404a-ac53-26f25100d2db service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:05:30.651180 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a790a87-7d15-404a-ac53-26f25100d2db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:05:30.658714 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a790a87-7d15-404a-ac53-26f25100d2db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:05:30.660123 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a790a87-7d15-404a-ac53-26f25100d2db service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:05:30.667678 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1198/2398] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:05:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:05:30.751187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3c644-1e87-46bd-9d54-ce78ef2185eb None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:05:30.751558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3c644-1e87-46bd-9d54-ce78ef2185eb None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:05:30.751840 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3c644-1e87-46bd-9d54-ce78ef2185eb None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:05:30.753210 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1201/2399] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 9 23:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 23:05:30.758903 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3984c-091e-4467-bb9d-741d1002b764 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:05:30.759181 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3984c-091e-4467-bb9d-741d1002b764 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:05:30.759453 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3984c-091e-4467-bb9d-741d1002b764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:05:30.780467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-4ab3984c-091e-4467-bb9d-741d1002b764 None None] MFA Rules not processed for user `02fcd7cc86134ce488c7dd75dfb174b8`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:05:30.781628 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-4ab3984c-091e-4467-bb9d-741d1002b764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:05:30.790752 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1199/2400] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 9 23:05:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:05:30.801037 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3c644-1e87-46bd-9d54-ce78ef2185eb None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:05:30.811711 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3c644-1e87-46bd-9d54-ce78ef2185eb None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:05:30.815235 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe961ee2-cd58-4c98-ac95-7c8c30275eb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:05:30.815857 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe961ee2-cd58-4c98-ac95-7c8c30275eb2 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:05:30.816136 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe961ee2-cd58-4c98-ac95-7c8c30275eb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:05:30.816411 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe961ee2-cd58-4c98-ac95-7c8c30275eb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:05:30.828034 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe961ee2-cd58-4c98-ac95-7c8c30275eb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:05:30.829453 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe961ee2-cd58-4c98-ac95-7c8c30275eb2 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:05:30.840315 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1202/2401] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:05:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 23:05:40.499172 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52c6c2-e667-4407-b76e-90c9caeabc59 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:05:40.499628 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52c6c2-e667-4407-b76e-90c9caeabc59 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:05:40.499878 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52c6c2-e667-4407-b76e-90c9caeabc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:05:40.517410 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-be52c6c2-e667-4407-b76e-90c9caeabc59 None None] MFA Rules not processed for user `1a6f50dcebf146baae8a9f0b5b112795`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:05:40.518689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-be52c6c2-e667-4407-b76e-90c9caeabc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:05:40.531386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1200/2402] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 9 23:05:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:05:40.538238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe961ee2-cd58-4c98-ac95-7c8c30275eb2 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:05:40.550044 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe961ee2-cd58-4c98-ac95-7c8c30275eb2 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:05:40.555254 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f032b6-05c1-43f7-942f-11a590f3561d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cb6c294ce1842b68a6bc73ac381ed6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:05:40.555883 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f032b6-05c1-43f7-942f-11a590f3561d service swift] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:05:40.556160 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f032b6-05c1-43f7-942f-11a590f3561d service swift] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:05:40.556432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f032b6-05c1-43f7-942f-11a590f3561d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:05:40.565487 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f032b6-05c1-43f7-942f-11a590f3561d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:05:40.567098 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f032b6-05c1-43f7-942f-11a590f3561d service swift] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:05:40.576028 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1203/2403] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 9 23:05:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 23:06:08.690793 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8e074-5737-4e7c-ae6a-e716e55da90e None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:06:08.691353 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8e074-5737-4e7c-ae6a-e716e55da90e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:06:08.691353 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8e074-5737-4e7c-ae6a-e716e55da90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:06:08.712847 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-7df8e074-5737-4e7c-ae6a-e716e55da90e None None] MFA Rules not processed for user `1a6f50dcebf146baae8a9f0b5b112795`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:06:08.714476 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7df8e074-5737-4e7c-ae6a-e716e55da90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:06:08.722732 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1201/2404] 10.222.0.23 () {58 vars in 967 bytes} [Mon May 9 23:06:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:06:08.737183 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f032b6-05c1-43f7-942f-11a590f3561d service swift] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:06:08.747954 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f032b6-05c1-43f7-942f-11a590f3561d service swift] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:06:08.753669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86940052-a6d1-4f48-aa52-2ddd74b32ac4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cb6c294ce1842b68a6bc73ac381ed6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:06:08.754569 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86940052-a6d1-4f48-aa52-2ddd74b32ac4 service swift] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:06:08.755915 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86940052-a6d1-4f48-aa52-2ddd74b32ac4 service swift] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:06:08.756206 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86940052-a6d1-4f48-aa52-2ddd74b32ac4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:06:08.766852 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86940052-a6d1-4f48-aa52-2ddd74b32ac4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:06:08.768679 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86940052-a6d1-4f48-aa52-2ddd74b32ac4 service swift] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:06:08.777670 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1204/2405] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 9 23:06:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 23:06:11.923123 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77488fa-c009-4717-9784-20508298f6dc None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:06:11.923659 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77488fa-c009-4717-9784-20508298f6dc None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:06:11.923659 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77488fa-c009-4717-9784-20508298f6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:06:11.941377 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e77488fa-c009-4717-9784-20508298f6dc None None] MFA Rules not processed for user `84a45506c19c4cda8173c164a097f925`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:06:11.942695 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e77488fa-c009-4717-9784-20508298f6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:06:11.997059 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1202/2406] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:06:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:06:17.149509 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7ca1f-0e87-472a-9f4c-24dbbb0a83dd None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:06:17.149882 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7ca1f-0e87-472a-9f4c-24dbbb0a83dd None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:06:17.150046 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7ca1f-0e87-472a-9f4c-24dbbb0a83dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:06:17.169308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e6d7ca1f-0e87-472a-9f4c-24dbbb0a83dd None None] MFA Rules not processed for user `7d3e164b111b4c3784b220ef483bb5b9`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:06:17.170617 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e6d7ca1f-0e87-472a-9f4c-24dbbb0a83dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:06:17.183139 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1205/2407] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:06:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:06:27.338289 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0135b630-b136-4b8e-9220-fc06082e997d None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:06:27.338872 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0135b630-b136-4b8e-9220-fc06082e997d None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:06:27.339127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0135b630-b136-4b8e-9220-fc06082e997d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:06:27.357991 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-0135b630-b136-4b8e-9220-fc06082e997d None None] MFA Rules not processed for user `e41b4feaa9be44e89210b70c19ef10ab`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:06:27.360074 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0135b630-b136-4b8e-9220-fc06082e997d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:06:27.420081 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1203/2408] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:06:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:07:08.321777 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2318675-b04b-476a-acb6-712c2bd862ac None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:08.322319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2318675-b04b-476a-acb6-712c2bd862ac None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:08.322417 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2318675-b04b-476a-acb6-712c2bd862ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:08.339467 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c2318675-b04b-476a-acb6-712c2bd862ac None None] MFA Rules not processed for user `c950754974d748519118d974492f15ce`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:07:08.340644 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c2318675-b04b-476a-acb6-712c2bd862ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:08.392707 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1206/2409] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:07:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:07:14.739491 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0135b630-b136-4b8e-9220-fc06082e997d None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:14.750216 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0135b630-b136-4b8e-9220-fc06082e997d None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:14.755108 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f25fd61-bdf2-435f-8b29-d7a7ed586fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:14.755993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f25fd61-bdf2-435f-8b29-d7a7ed586fc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:14.756269 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f25fd61-bdf2-435f-8b29-d7a7ed586fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:14.756539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f25fd61-bdf2-435f-8b29-d7a7ed586fc4 admin admin] PATH_INFO: `/v3/users/7d3e164b111b4c3784b220ef483bb5b9` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:14.757707 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f25fd61-bdf2-435f-8b29-d7a7ed586fc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d3e164b111b4c3784b220ef483bb5b9)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:14.759663 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f25fd61-bdf2-435f-8b29-d7a7ed586fc4 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:14.807387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-2f25fd61-bdf2-435f-8b29-d7a7ed586fc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d3e164b111b4c3784b220ef483bb5b9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:14.813916 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-2f25fd61-bdf2-435f-8b29-d7a7ed586fc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d3e164b111b4c3784b220ef483bb5b9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:14.814602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-2f25fd61-bdf2-435f-8b29-d7a7ed586fc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d3e164b111b4c3784b220ef483bb5b9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:14.822359 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-2f25fd61-bdf2-435f-8b29-d7a7ed586fc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d3e164b111b4c3784b220ef483bb5b9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:14.822980 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-2f25fd61-bdf2-435f-8b29-d7a7ed586fc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d3e164b111b4c3784b220ef483bb5b9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:14.841219 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1204/2410] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:07:14 2022] DELETE /identity/v3/users/7d3e164b111b4c3784b220ef483bb5b9 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:07:15.014996 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2318675-b04b-476a-acb6-712c2bd862ac None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:15.030289 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2318675-b04b-476a-acb6-712c2bd862ac None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:15.094359 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14ad69-0915-4226-b315-8c419f297218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:15.095292 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14ad69-0915-4226-b315-8c419f297218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:15.095574 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14ad69-0915-4226-b315-8c419f297218 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:15.095843 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14ad69-0915-4226-b315-8c419f297218 admin admin] PATH_INFO: `/v3/projects/9a54785a57d74beb87b5a31f928583e7` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:15.097117 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc14ad69-0915-4226-b315-8c419f297218 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a54785a57d74beb87b5a31f928583e7)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:15.099072 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc14ad69-0915-4226-b315-8c419f297218 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:15.158221 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-bc14ad69-0915-4226-b315-8c419f297218 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a54785a57d74beb87b5a31f928583e7', 'request_id': 'req-bc14ad69-0915-4226-b315-8c419f297218'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:15.163550 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-bc14ad69-0915-4226-b315-8c419f297218 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a54785a57d74beb87b5a31f928583e7', 'request_id': 'req-bc14ad69-0915-4226-b315-8c419f297218'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:15.169726 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-bc14ad69-0915-4226-b315-8c419f297218 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a54785a57d74beb87b5a31f928583e7', 'request_id': 'req-bc14ad69-0915-4226-b315-8c419f297218'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:15.182332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-bc14ad69-0915-4226-b315-8c419f297218 admin admin] The token cache is being invalidate because project 9a54785a57d74beb87b5a31f928583e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:07:15.182622 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-bc14ad69-0915-4226-b315-8c419f297218 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:15.183629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-bc14ad69-0915-4226-b315-8c419f297218 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:15.185119 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1207/2411] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:07:15 2022] DELETE /identity/v3/projects/9a54785a57d74beb87b5a31f928583e7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:07:20.632895 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f25fd61-bdf2-435f-8b29-d7a7ed586fc4 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:20.640113 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2f25fd61-bdf2-435f-8b29-d7a7ed586fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:20.716312 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f25fd61-bdf2-435f-8b29-d7a7ed586fc4 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:20.779566 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea33c67-3273-4e36-9b0a-314d264f9862 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:20.780360 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea33c67-3273-4e36-9b0a-314d264f9862 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:20.780708 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea33c67-3273-4e36-9b0a-314d264f9862 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:20.781051 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea33c67-3273-4e36-9b0a-314d264f9862 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:20.783133 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8ea33c67-3273-4e36-9b0a-314d264f9862 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:20.860177 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea33c67-3273-4e36-9b0a-314d264f9862 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:20.861622 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea33c67-3273-4e36-9b0a-314d264f9862 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:20.873373 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1205/2412] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:07:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:07:25.607186 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14ad69-0915-4226-b315-8c419f297218 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:25.608591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-bc14ad69-0915-4226-b315-8c419f297218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:25.666700 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14ad69-0915-4226-b315-8c419f297218 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:25.670237 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de84fb5-9699-4515-8439-2351fa5494ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:25.670875 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de84fb5-9699-4515-8439-2351fa5494ac service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:25.671151 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de84fb5-9699-4515-8439-2351fa5494ac service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:25.671423 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de84fb5-9699-4515-8439-2351fa5494ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:25.673787 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6de84fb5-9699-4515-8439-2351fa5494ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:25.730821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de84fb5-9699-4515-8439-2351fa5494ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:25.732510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de84fb5-9699-4515-8439-2351fa5494ac service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:25.742051 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1208/2413] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:07:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:07:29.510743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea33c67-3273-4e36-9b0a-314d264f9862 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:29.512268 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8ea33c67-3273-4e36-9b0a-314d264f9862 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:29.522602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea33c67-3273-4e36-9b0a-314d264f9862 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:29.526328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91e266-79f9-42ce-a4e0-5492c113e9d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:29.526997 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91e266-79f9-42ce-a4e0-5492c113e9d6 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:29.527276 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91e266-79f9-42ce-a4e0-5492c113e9d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:29.527565 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91e266-79f9-42ce-a4e0-5492c113e9d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:29.529274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2c91e266-79f9-42ce-a4e0-5492c113e9d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:29.542397 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91e266-79f9-42ce-a4e0-5492c113e9d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:29.543799 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91e266-79f9-42ce-a4e0-5492c113e9d6 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:29.554928 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1206/2414] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:07:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:07:30.552204 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de84fb5-9699-4515-8439-2351fa5494ac service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:30.561219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de84fb5-9699-4515-8439-2351fa5494ac service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:30.564712 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b5ebc-eb5e-4628-abfb-4c8ac1a3ee7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:30.565335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b5ebc-eb5e-4628-abfb-4c8ac1a3ee7f service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:30.565610 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b5ebc-eb5e-4628-abfb-4c8ac1a3ee7f service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:30.565881 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b5ebc-eb5e-4628-abfb-4c8ac1a3ee7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:30.567651 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-553b5ebc-eb5e-4628-abfb-4c8ac1a3ee7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:30.627976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b5ebc-eb5e-4628-abfb-4c8ac1a3ee7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:30.628997 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b5ebc-eb5e-4628-abfb-4c8ac1a3ee7f service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:30.638281 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1209/2415] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:07:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:07:33.083132 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3592a6-633b-47dc-a430-35ec808e0431 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:33.083579 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3592a6-633b-47dc-a430-35ec808e0431 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:33.083675 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3592a6-633b-47dc-a430-35ec808e0431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:33.101367 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-1e3592a6-633b-47dc-a430-35ec808e0431 None None] MFA Rules not processed for user `1a6f50dcebf146baae8a9f0b5b112795`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:07:33.102596 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1e3592a6-633b-47dc-a430-35ec808e0431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:33.156864 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1207/2416] 10.222.0.23 () {58 vars in 967 bytes} [Mon May 9 23:07:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:07:33.215971 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b5ebc-eb5e-4628-abfb-4c8ac1a3ee7f service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:33.217386 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-553b5ebc-eb5e-4628-abfb-4c8ac1a3ee7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:33.285567 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b5ebc-eb5e-4628-abfb-4c8ac1a3ee7f service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:33.290603 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb9b8f-2bf5-45c7-90d1-325f3ed5f973 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cb6c294ce1842b68a6bc73ac381ed6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:33.291407 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb9b8f-2bf5-45c7-90d1-325f3ed5f973 service swift] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:33.291694 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb9b8f-2bf5-45c7-90d1-325f3ed5f973 service swift] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:33.291971 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb9b8f-2bf5-45c7-90d1-325f3ed5f973 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:33.300721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb9b8f-2bf5-45c7-90d1-325f3ed5f973 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:33.302327 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb9b8f-2bf5-45c7-90d1-325f3ed5f973 service swift] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:33.310365 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1210/2417] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 9 23:07:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 23:07:40.179761 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3592a6-633b-47dc-a430-35ec808e0431 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:40.182704 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1e3592a6-633b-47dc-a430-35ec808e0431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:40.196970 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3592a6-633b-47dc-a430-35ec808e0431 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:40.201179 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ad349-8dbd-408a-b9ad-eed16609f2b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:40.202003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ad349-8dbd-408a-b9ad-eed16609f2b8 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:40.202529 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ad349-8dbd-408a-b9ad-eed16609f2b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:40.203613 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ad349-8dbd-408a-b9ad-eed16609f2b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:40.205618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c82ad349-8dbd-408a-b9ad-eed16609f2b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:40.291329 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ad349-8dbd-408a-b9ad-eed16609f2b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:40.294925 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ad349-8dbd-408a-b9ad-eed16609f2b8 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:40.310743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1208/2418] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:07:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:07:41.547061 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb9b8f-2bf5-45c7-90d1-325f3ed5f973 service swift] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:41.548375 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-2aeb9b8f-2bf5-45c7-90d1-325f3ed5f973 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:41.569204 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb9b8f-2bf5-45c7-90d1-325f3ed5f973 service swift] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:41.575081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebb26c-2e6c-4840-bf83-0eb0375540c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:41.576141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebb26c-2e6c-4840-bf83-0eb0375540c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:41.576478 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebb26c-2e6c-4840-bf83-0eb0375540c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:41.576812 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebb26c-2e6c-4840-bf83-0eb0375540c2 admin admin] PATH_INFO: `/v3/users/19fbe84526dd48a6be14c1aebf5713c0` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:41.579546 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebb26c-2e6c-4840-bf83-0eb0375540c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19fbe84526dd48a6be14c1aebf5713c0)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:41.581766 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebb26c-2e6c-4840-bf83-0eb0375540c2 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:41.654602 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b1ebb26c-2e6c-4840-bf83-0eb0375540c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19fbe84526dd48a6be14c1aebf5713c0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:41.663450 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b1ebb26c-2e6c-4840-bf83-0eb0375540c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19fbe84526dd48a6be14c1aebf5713c0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:41.673021 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b1ebb26c-2e6c-4840-bf83-0eb0375540c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19fbe84526dd48a6be14c1aebf5713c0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:41.673755 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b1ebb26c-2e6c-4840-bf83-0eb0375540c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19fbe84526dd48a6be14c1aebf5713c0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:41.687509 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b1ebb26c-2e6c-4840-bf83-0eb0375540c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19fbe84526dd48a6be14c1aebf5713c0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:41.693410 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1211/2419] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:07:41 2022] DELETE /identity/v3/users/19fbe84526dd48a6be14c1aebf5713c0 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:07:41.867721 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ad349-8dbd-408a-b9ad-eed16609f2b8 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:41.878812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ad349-8dbd-408a-b9ad-eed16609f2b8 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:41.935750 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56333cad-2df9-4a8b-a163-9674cf88b775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:41.936612 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56333cad-2df9-4a8b-a163-9674cf88b775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:41.937397 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56333cad-2df9-4a8b-a163-9674cf88b775 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:41.937715 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56333cad-2df9-4a8b-a163-9674cf88b775 admin admin] PATH_INFO: `/v3/projects/93df4f41ba52453dabfd21f82cb9238b` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:41.939181 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56333cad-2df9-4a8b-a163-9674cf88b775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93df4f41ba52453dabfd21f82cb9238b)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:41.941311 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56333cad-2df9-4a8b-a163-9674cf88b775 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:41.989038 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-56333cad-2df9-4a8b-a163-9674cf88b775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93df4f41ba52453dabfd21f82cb9238b', 'request_id': 'req-56333cad-2df9-4a8b-a163-9674cf88b775'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:41.996483 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-56333cad-2df9-4a8b-a163-9674cf88b775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93df4f41ba52453dabfd21f82cb9238b', 'request_id': 'req-56333cad-2df9-4a8b-a163-9674cf88b775'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:42.002761 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-56333cad-2df9-4a8b-a163-9674cf88b775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93df4f41ba52453dabfd21f82cb9238b', 'request_id': 'req-56333cad-2df9-4a8b-a163-9674cf88b775'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:42.015596 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-56333cad-2df9-4a8b-a163-9674cf88b775 admin admin] The token cache is being invalidate because project 93df4f41ba52453dabfd21f82cb9238b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:07:42.015952 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-56333cad-2df9-4a8b-a163-9674cf88b775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:42.016549 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-56333cad-2df9-4a8b-a163-9674cf88b775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:42.018105 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1209/2420] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:07:41 2022] DELETE /identity/v3/projects/93df4f41ba52453dabfd21f82cb9238b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:07:42.022319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebb26c-2e6c-4840-bf83-0eb0375540c2 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:42.023785 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b1ebb26c-2e6c-4840-bf83-0eb0375540c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:42.037224 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebb26c-2e6c-4840-bf83-0eb0375540c2 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:42.096926 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7306f80-acb9-434e-8ede-39ecf351e076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:42.098514 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7306f80-acb9-434e-8ede-39ecf351e076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:42.098811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7306f80-acb9-434e-8ede-39ecf351e076 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:42.100010 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7306f80-acb9-434e-8ede-39ecf351e076 admin admin] PATH_INFO: `/v3/users/ea3b7736ecf041a487b9e983f3fc4e81` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:42.101267 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7306f80-acb9-434e-8ede-39ecf351e076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea3b7736ecf041a487b9e983f3fc4e81)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:42.103260 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea778883-c457-43d1-8d6d-f269140e7d87 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:42.103545 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea778883-c457-43d1-8d6d-f269140e7d87 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:42.103815 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea778883-c457-43d1-8d6d-f269140e7d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:42.104191 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7306f80-acb9-434e-8ede-39ecf351e076 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:42.123299 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ea778883-c457-43d1-8d6d-f269140e7d87 None None] MFA Rules not processed for user `84a45506c19c4cda8173c164a097f925`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:07:42.125744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ea778883-c457-43d1-8d6d-f269140e7d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:42.163120 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d7306f80-acb9-434e-8ede-39ecf351e076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea3b7736ecf041a487b9e983f3fc4e81'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:42.175258 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d7306f80-acb9-434e-8ede-39ecf351e076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea3b7736ecf041a487b9e983f3fc4e81'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:42.186203 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d7306f80-acb9-434e-8ede-39ecf351e076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea3b7736ecf041a487b9e983f3fc4e81'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:42.187652 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d7306f80-acb9-434e-8ede-39ecf351e076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea3b7736ecf041a487b9e983f3fc4e81'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:42.196670 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1210/2421] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:07:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:07:42.203831 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d7306f80-acb9-434e-8ede-39ecf351e076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea3b7736ecf041a487b9e983f3fc4e81'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:42.214778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1212/2422] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:07:42 2022] DELETE /identity/v3/users/ea3b7736ecf041a487b9e983f3fc4e81 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:07:42.544893 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea778883-c457-43d1-8d6d-f269140e7d87 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:42.546718 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ea778883-c457-43d1-8d6d-f269140e7d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:42.562803 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea778883-c457-43d1-8d6d-f269140e7d87 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:42.627439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7306f80-acb9-434e-8ede-39ecf351e076 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:42.629369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d7306f80-acb9-434e-8ede-39ecf351e076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:42.642548 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311a669-47bd-4f9d-ac5a-3500c1ae25a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:42.643449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311a669-47bd-4f9d-ac5a-3500c1ae25a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:42.643735 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311a669-47bd-4f9d-ac5a-3500c1ae25a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:42.644011 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311a669-47bd-4f9d-ac5a-3500c1ae25a8 admin admin] PATH_INFO: `/v3/projects/3d9e04b82a614f86be6d631e4bdc6918` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:42.645244 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e311a669-47bd-4f9d-ac5a-3500c1ae25a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d9e04b82a614f86be6d631e4bdc6918)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:42.647270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e311a669-47bd-4f9d-ac5a-3500c1ae25a8 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:42.696440 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e311a669-47bd-4f9d-ac5a-3500c1ae25a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d9e04b82a614f86be6d631e4bdc6918', 'request_id': 'req-e311a669-47bd-4f9d-ac5a-3500c1ae25a8'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:42.704600 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e311a669-47bd-4f9d-ac5a-3500c1ae25a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d9e04b82a614f86be6d631e4bdc6918', 'request_id': 'req-e311a669-47bd-4f9d-ac5a-3500c1ae25a8'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:42.711190 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e311a669-47bd-4f9d-ac5a-3500c1ae25a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d9e04b82a614f86be6d631e4bdc6918', 'request_id': 'req-e311a669-47bd-4f9d-ac5a-3500c1ae25a8'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:42.712577 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7306f80-acb9-434e-8ede-39ecf351e076 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:42.727347 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e311a669-47bd-4f9d-ac5a-3500c1ae25a8 admin admin] The token cache is being invalidate because project 3d9e04b82a614f86be6d631e4bdc6918 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:07:42.727770 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e311a669-47bd-4f9d-ac5a-3500c1ae25a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:42.728652 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e311a669-47bd-4f9d-ac5a-3500c1ae25a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:42.730944 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1211/2423] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:07:42 2022] DELETE /identity/v3/projects/3d9e04b82a614f86be6d631e4bdc6918 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:07:42.768545 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdbe1e4-3749-499a-adc0-fe570dc7298a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:42.769509 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbe1e4-3749-499a-adc0-fe570dc7298a service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:42.770189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbe1e4-3749-499a-adc0-fe570dc7298a service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:42.770633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbe1e4-3749-499a-adc0-fe570dc7298a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:42.772827 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7cdbe1e4-3749-499a-adc0-fe570dc7298a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:42.783509 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdbe1e4-3749-499a-adc0-fe570dc7298a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:42.786811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdbe1e4-3749-499a-adc0-fe570dc7298a service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:42.804273 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1213/2424] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:07:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:07:43.298223 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311a669-47bd-4f9d-ac5a-3500c1ae25a8 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:43.300156 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e311a669-47bd-4f9d-ac5a-3500c1ae25a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:43.379107 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311a669-47bd-4f9d-ac5a-3500c1ae25a8 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:43.385694 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdefe9b7-f41a-470a-a1e2-12509f927fae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf9bc2fa959d4706b9f9e87509829059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:43.386314 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefe9b7-f41a-470a-a1e2-12509f927fae service placement] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:43.386589 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefe9b7-f41a-470a-a1e2-12509f927fae service placement] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:43.386869 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefe9b7-f41a-470a-a1e2-12509f927fae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:43.389698 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-cdefe9b7-f41a-470a-a1e2-12509f927fae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:43.405061 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdefe9b7-f41a-470a-a1e2-12509f927fae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:43.407387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdefe9b7-f41a-470a-a1e2-12509f927fae service placement] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:43.426405 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1212/2425] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:07:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:07:44.869820 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdbe1e4-3749-499a-adc0-fe570dc7298a service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:44.871514 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7cdbe1e4-3749-499a-adc0-fe570dc7298a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:44.939603 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdbe1e4-3749-499a-adc0-fe570dc7298a service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:44.944131 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f959d-3d4d-4df1-a62d-2eb1ef97cb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:44.944983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f959d-3d4d-4df1-a62d-2eb1ef97cb21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:44.945257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f959d-3d4d-4df1-a62d-2eb1ef97cb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:44.945526 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f959d-3d4d-4df1-a62d-2eb1ef97cb21 admin admin] PATH_INFO: `/v3/users/e41b4feaa9be44e89210b70c19ef10ab` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:44.946777 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f959d-3d4d-4df1-a62d-2eb1ef97cb21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e41b4feaa9be44e89210b70c19ef10ab)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:44.948716 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f959d-3d4d-4df1-a62d-2eb1ef97cb21 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:44.993882 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-fe0f959d-3d4d-4df1-a62d-2eb1ef97cb21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e41b4feaa9be44e89210b70c19ef10ab'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:45.001063 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-fe0f959d-3d4d-4df1-a62d-2eb1ef97cb21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e41b4feaa9be44e89210b70c19ef10ab'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:45.007983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-fe0f959d-3d4d-4df1-a62d-2eb1ef97cb21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e41b4feaa9be44e89210b70c19ef10ab'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:45.008636 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-fe0f959d-3d4d-4df1-a62d-2eb1ef97cb21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e41b4feaa9be44e89210b70c19ef10ab'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:45.020788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-fe0f959d-3d4d-4df1-a62d-2eb1ef97cb21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e41b4feaa9be44e89210b70c19ef10ab'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:45.027094 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1214/2426] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:07:44 2022] DELETE /identity/v3/users/e41b4feaa9be44e89210b70c19ef10ab => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:07:45.201386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdefe9b7-f41a-470a-a1e2-12509f927fae service placement] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:45.209738 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f959d-3d4d-4df1-a62d-2eb1ef97cb21 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:45.211715 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-fe0f959d-3d4d-4df1-a62d-2eb1ef97cb21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:45.219925 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdefe9b7-f41a-470a-a1e2-12509f927fae service placement] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:45.278572 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f959d-3d4d-4df1-a62d-2eb1ef97cb21 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:45.282964 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb83862-60c1-465f-914f-9be97d7fa295 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:45.283609 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb83862-60c1-465f-914f-9be97d7fa295 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:45.283884 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb83862-60c1-465f-914f-9be97d7fa295 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:45.284155 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb83862-60c1-465f-914f-9be97d7fa295 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:45.285935 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4fb83862-60c1-465f-914f-9be97d7fa295 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:45.291276 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb6090-1bff-463e-b587-487d7c1f6e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:45.292135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb6090-1bff-463e-b587-487d7c1f6e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:45.292436 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb6090-1bff-463e-b587-487d7c1f6e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:45.292710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb6090-1bff-463e-b587-487d7c1f6e2b admin admin] PATH_INFO: `/v3/projects/0cb055ff9f0d4af6962070c7a2f19bf3` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:45.294533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb6090-1bff-463e-b587-487d7c1f6e2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cb055ff9f0d4af6962070c7a2f19bf3)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:45.296509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb6090-1bff-463e-b587-487d7c1f6e2b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:45.299110 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb83862-60c1-465f-914f-9be97d7fa295 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:45.300728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb83862-60c1-465f-914f-9be97d7fa295 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:45.310555 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1215/2427] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:07:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 23:07:45.347273 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-44eb6090-1bff-463e-b587-487d7c1f6e2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cb055ff9f0d4af6962070c7a2f19bf3', 'request_id': 'req-44eb6090-1bff-463e-b587-487d7c1f6e2b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:45.353215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-44eb6090-1bff-463e-b587-487d7c1f6e2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cb055ff9f0d4af6962070c7a2f19bf3', 'request_id': 'req-44eb6090-1bff-463e-b587-487d7c1f6e2b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:45.359214 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-44eb6090-1bff-463e-b587-487d7c1f6e2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cb055ff9f0d4af6962070c7a2f19bf3', 'request_id': 'req-44eb6090-1bff-463e-b587-487d7c1f6e2b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:45.371220 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-44eb6090-1bff-463e-b587-487d7c1f6e2b admin admin] The token cache is being invalidate because project 0cb055ff9f0d4af6962070c7a2f19bf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:07:45.371500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-44eb6090-1bff-463e-b587-487d7c1f6e2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:45.372110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-44eb6090-1bff-463e-b587-487d7c1f6e2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:07:45.373635 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1213/2428] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:07:45 2022] DELETE /identity/v3/projects/0cb055ff9f0d4af6962070c7a2f19bf3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:07:45.378608 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28a462-458b-4932-b910-c6937458038c None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:45.378892 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28a462-458b-4932-b910-c6937458038c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:45.379165 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28a462-458b-4932-b910-c6937458038c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:45.396965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-ac28a462-458b-4932-b910-c6937458038c None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:07:45.398092 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ac28a462-458b-4932-b910-c6937458038c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:45.474496 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1216/2429] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:07:45.482389 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f3a48-b610-4acb-a6b4-9612c3272a6f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:45.482690 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f3a48-b610-4acb-a6b4-9612c3272a6f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:45.482966 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f3a48-b610-4acb-a6b4-9612c3272a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:45.505156 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-848f3a48-b610-4acb-a6b4-9612c3272a6f None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:07:45.506492 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-848f3a48-b610-4acb-a6b4-9612c3272a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:45.516482 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1214/2430] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:07:45.521385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28a462-458b-4932-b910-c6937458038c None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:45.535990 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28a462-458b-4932-b910-c6937458038c None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:45.542433 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070e2fb-f128-4a53-a2c3-81ea2b1e8770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:45.543122 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070e2fb-f128-4a53-a2c3-81ea2b1e8770 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:45.543406 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070e2fb-f128-4a53-a2c3-81ea2b1e8770 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:45.543686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070e2fb-f128-4a53-a2c3-81ea2b1e8770 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:45.544477 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070e2fb-f128-4a53-a2c3-81ea2b1e8770 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:07:45.544788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070e2fb-f128-4a53-a2c3-81ea2b1e8770 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:45.546855 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070e2fb-f128-4a53-a2c3-81ea2b1e8770 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:45.561057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1217/2431] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 23:07:45.565542 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f3a48-b610-4acb-a6b4-9612c3272a6f None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:45.583094 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f3a48-b610-4acb-a6b4-9612c3272a6f None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:45.590703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a4bec-76f5-4c7b-9de7-2a3256f135de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:45.590703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a4bec-76f5-4c7b-9de7-2a3256f135de admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:45.590703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a4bec-76f5-4c7b-9de7-2a3256f135de admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:45.590703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a4bec-76f5-4c7b-9de7-2a3256f135de admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:45.591157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a4bec-76f5-4c7b-9de7-2a3256f135de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:45.592131 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a4bec-76f5-4c7b-9de7-2a3256f135de admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:45.629308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1215/2432] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:07:45 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:07:45.630539 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070e2fb-f128-4a53-a2c3-81ea2b1e8770 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:45.644167 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070e2fb-f128-4a53-a2c3-81ea2b1e8770 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:45.704936 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2db30b-5bd7-4d36-ab60-d2917c13145e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:45.705858 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2db30b-5bd7-4d36-ab60-d2917c13145e admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:45.706093 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2db30b-5bd7-4d36-ab60-d2917c13145e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:45.706496 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2db30b-5bd7-4d36-ab60-d2917c13145e admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:45.707307 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2db30b-5bd7-4d36-ab60-d2917c13145e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:45.709256 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2db30b-5bd7-4d36-ab60-d2917c13145e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:45.729359 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1218/2433] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:07:45 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:07:45.734313 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a4bec-76f5-4c7b-9de7-2a3256f135de admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:45.745201 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a4bec-76f5-4c7b-9de7-2a3256f135de admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:45.750183 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847174be-3b44-4a46-8606-30760727f1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:45.750844 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847174be-3b44-4a46-8606-30760727f1fe admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:45.751126 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847174be-3b44-4a46-8606-30760727f1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:45.751415 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847174be-3b44-4a46-8606-30760727f1fe admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:45.752157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847174be-3b44-4a46-8606-30760727f1fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:45.753748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847174be-3b44-4a46-8606-30760727f1fe admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:45.764156 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1216/2434] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:07:45.769167 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2db30b-5bd7-4d36-ab60-d2917c13145e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:45.778979 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2db30b-5bd7-4d36-ab60-d2917c13145e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:45.785501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f19a2-6c76-424e-b670-ecdf97b32191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:45.786254 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f19a2-6c76-424e-b670-ecdf97b32191 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:45.786465 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f19a2-6c76-424e-b670-ecdf97b32191 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:45.786740 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f19a2-6c76-424e-b670-ecdf97b32191 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:45.787452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f19a2-6c76-424e-b670-ecdf97b32191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:45.789044 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f19a2-6c76-424e-b670-ecdf97b32191 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:45.800193 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1219/2435] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:07:45.805660 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847174be-3b44-4a46-8606-30760727f1fe admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:45.817540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847174be-3b44-4a46-8606-30760727f1fe admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:45.822556 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01831d7a-baa8-43b6-907a-0f3fc76568a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:45.823448 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01831d7a-baa8-43b6-907a-0f3fc76568a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:45.823721 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01831d7a-baa8-43b6-907a-0f3fc76568a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:45.824021 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01831d7a-baa8-43b6-907a-0f3fc76568a3 admin admin] PATH_INFO: `/v3/projects/c0df002b4f614f4bbac1782e1fbb484a/users/1de1f7ce5e5f4cfbb283f0c79d87f9c8/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:45.845538 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01831d7a-baa8-43b6-907a-0f3fc76568a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0df002b4f614f4bbac1782e1fbb484a, user_id=1de1f7ce5e5f4cfbb283f0c79d87f9c8, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:45.848438 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01831d7a-baa8-43b6-907a-0f3fc76568a3 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:45.866234 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1217/2436] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:07:45 2022] PUT /identity/v3/projects/c0df002b4f614f4bbac1782e1fbb484a/users/1de1f7ce5e5f4cfbb283f0c79d87f9c8/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:07:45.868045 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af81052-02fa-4aa6-b381-47a276cb02da None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:45.868364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af81052-02fa-4aa6-b381-47a276cb02da None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:45.868691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af81052-02fa-4aa6-b381-47a276cb02da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:45.890327 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-9af81052-02fa-4aa6-b381-47a276cb02da None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:07:45.891561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9af81052-02fa-4aa6-b381-47a276cb02da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:45.953698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1220/2437] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:07:45.963501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01831d7a-baa8-43b6-907a-0f3fc76568a3 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:45.965408 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-01831d7a-baa8-43b6-907a-0f3fc76568a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:46.042794 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01831d7a-baa8-43b6-907a-0f3fc76568a3 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:46.047132 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cfec2-c34b-4aef-bda5-82f9f83d84ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:46.047832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cfec2-c34b-4aef-bda5-82f9f83d84ed service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:46.048746 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cfec2-c34b-4aef-bda5-82f9f83d84ed service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:46.049026 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cfec2-c34b-4aef-bda5-82f9f83d84ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:46.059284 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626cfec2-c34b-4aef-bda5-82f9f83d84ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:46.060738 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626cfec2-c34b-4aef-bda5-82f9f83d84ed service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:46.092911 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1218/2438] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:07:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:07:50.316289 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283acfa9-8c24-434c-a9a0-e7beabd3e1ff None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:50.316779 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283acfa9-8c24-434c-a9a0-e7beabd3e1ff None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:50.316885 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283acfa9-8c24-434c-a9a0-e7beabd3e1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:50.345380 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-283acfa9-8c24-434c-a9a0-e7beabd3e1ff None None] MFA Rules not processed for user `1de1f7ce5e5f4cfbb283f0c79d87f9c8`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:07:50.346642 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-283acfa9-8c24-434c-a9a0-e7beabd3e1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:50.420411 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1221/2439] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:07:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:07:50.435301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cfec2-c34b-4aef-bda5-82f9f83d84ed service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:50.438312 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-626cfec2-c34b-4aef-bda5-82f9f83d84ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:50.516526 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cfec2-c34b-4aef-bda5-82f9f83d84ed service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:50.522537 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0b6f1-3f2e-4174-b288-6297bf258076 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:50.523466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0b6f1-3f2e-4174-b288-6297bf258076 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:50.523466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0b6f1-3f2e-4174-b288-6297bf258076 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:50.523741 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0b6f1-3f2e-4174-b288-6297bf258076 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:50.536922 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0b6f1-3f2e-4174-b288-6297bf258076 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:50.538868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0b6f1-3f2e-4174-b288-6297bf258076 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:50.548936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1219/2440] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:07:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:07:51.906733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5518c7c-a368-4493-97aa-82ec6a49e276 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:51.907308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5518c7c-a368-4493-97aa-82ec6a49e276 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:51.907308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5518c7c-a368-4493-97aa-82ec6a49e276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:51.929088 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f5518c7c-a368-4493-97aa-82ec6a49e276 None None] MFA Rules not processed for user `1de1f7ce5e5f4cfbb283f0c79d87f9c8`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:07:51.930343 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f5518c7c-a368-4493-97aa-82ec6a49e276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:51.941930 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1222/2441] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:07:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:07:55.924130 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0b6f1-3f2e-4174-b288-6297bf258076 service nova] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:07:55.925562 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-fed0b6f1-3f2e-4174-b288-6297bf258076 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:55.941834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0b6f1-3f2e-4174-b288-6297bf258076 service nova] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:07:55.949821 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8d0fb-ce00-468c-ad21-6a31ff02add2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:07:55.950509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8d0fb-ce00-468c-ad21-6a31ff02add2 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:07:55.950941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8d0fb-ce00-468c-ad21-6a31ff02add2 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:07:55.951221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8d0fb-ce00-468c-ad21-6a31ff02add2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:07:55.953668 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3bf8d0fb-ce00-468c-ad21-6a31ff02add2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:07:55.971431 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8d0fb-ce00-468c-ad21-6a31ff02add2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:07:55.973035 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8d0fb-ce00-468c-ad21-6a31ff02add2 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:07:55.990786 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1220/2442] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:07:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:08:01.268077 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5518c7c-a368-4493-97aa-82ec6a49e276 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:08:01.281578 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5518c7c-a368-4493-97aa-82ec6a49e276 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:08:01.287093 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24c9fa-7657-4390-9434-55409b720051 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:08:01.287736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24c9fa-7657-4390-9434-55409b720051 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:08:01.288018 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24c9fa-7657-4390-9434-55409b720051 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:08:01.288294 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24c9fa-7657-4390-9434-55409b720051 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:08:01.290077 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-2f24c9fa-7657-4390-9434-55409b720051 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:08:01.302938 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f24c9fa-7657-4390-9434-55409b720051 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:08:01.304516 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f24c9fa-7657-4390-9434-55409b720051 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:08:01.314845 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1223/2443] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:08:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:08:20.655846 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a58990-c325-416e-bcaa-1065fd574f34 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:08:20.656318 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a58990-c325-416e-bcaa-1065fd574f34 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:08:20.656423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a58990-c325-416e-bcaa-1065fd574f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:08:20.673725 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-03a58990-c325-416e-bcaa-1065fd574f34 None None] MFA Rules not processed for user `1a6f50dcebf146baae8a9f0b5b112795`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:08:20.674846 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-03a58990-c325-416e-bcaa-1065fd574f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:08:20.734094 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1221/2444] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 9 23:08:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:08:20.746468 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24c9fa-7657-4390-9434-55409b720051 service nova] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:08:20.747914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-2f24c9fa-7657-4390-9434-55409b720051 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:08:20.818399 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24c9fa-7657-4390-9434-55409b720051 service nova] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:08:20.823997 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf42d60-d25a-4404-9afe-0d00bcf3565c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cb6c294ce1842b68a6bc73ac381ed6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:08:20.824659 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf42d60-d25a-4404-9afe-0d00bcf3565c service swift] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:08:20.824944 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf42d60-d25a-4404-9afe-0d00bcf3565c service swift] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:08:20.825225 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf42d60-d25a-4404-9afe-0d00bcf3565c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:08:20.834338 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf42d60-d25a-4404-9afe-0d00bcf3565c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:08:20.836205 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf42d60-d25a-4404-9afe-0d00bcf3565c service swift] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:08:20.844451 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1224/2445] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 9 23:08:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 09 23:08:35.262532 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e1e16-a0a0-4c99-8bd3-4d70938995aa None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:08:35.262989 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e1e16-a0a0-4c99-8bd3-4d70938995aa None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:08:35.263100 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e1e16-a0a0-4c99-8bd3-4d70938995aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:08:35.281523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-263e1e16-a0a0-4c99-8bd3-4d70938995aa None None] MFA Rules not processed for user `1a6f50dcebf146baae8a9f0b5b112795`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:08:35.282746 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-263e1e16-a0a0-4c99-8bd3-4d70938995aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:08:35.290017 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1222/2446] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 9 23:08:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:08:35.311575 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf42d60-d25a-4404-9afe-0d00bcf3565c service swift] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:08:35.322038 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf42d60-d25a-4404-9afe-0d00bcf3565c service swift] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:08:35.326944 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50d0e1-9820-4cf7-92f5-2d1848c57035 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cb6c294ce1842b68a6bc73ac381ed6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:08:35.328153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50d0e1-9820-4cf7-92f5-2d1848c57035 service swift] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:08:35.328435 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50d0e1-9820-4cf7-92f5-2d1848c57035 service swift] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:08:35.328704 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50d0e1-9820-4cf7-92f5-2d1848c57035 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:08:35.336970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee50d0e1-9820-4cf7-92f5-2d1848c57035 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:08:35.338765 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee50d0e1-9820-4cf7-92f5-2d1848c57035 service swift] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:08:35.347270 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1225/2447] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 9 23:08:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 23:08:43.035003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e1e16-a0a0-4c99-8bd3-4d70938995aa None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:08:43.050054 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e1e16-a0a0-4c99-8bd3-4d70938995aa None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:08:43.060454 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac7b1e-2428-423c-a717-5281c6fdeef1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:08:43.061160 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac7b1e-2428-423c-a717-5281c6fdeef1 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:08:43.061564 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac7b1e-2428-423c-a717-5281c6fdeef1 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:08:43.061856 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac7b1e-2428-423c-a717-5281c6fdeef1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:08:43.064623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d8ac7b1e-2428-423c-a717-5281c6fdeef1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:08:43.132815 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac7b1e-2428-423c-a717-5281c6fdeef1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:08:43.134503 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac7b1e-2428-423c-a717-5281c6fdeef1 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:08:43.144709 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1223/2448] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:08:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:08:51.116198 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00824aee-91dc-4ca7-ada9-05fe7f854d08 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:08:51.116758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00824aee-91dc-4ca7-ada9-05fe7f854d08 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:08:51.116758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00824aee-91dc-4ca7-ada9-05fe7f854d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:08:51.134273 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-00824aee-91dc-4ca7-ada9-05fe7f854d08 None None] MFA Rules not processed for user `1a6f50dcebf146baae8a9f0b5b112795`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:08:51.135500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-00824aee-91dc-4ca7-ada9-05fe7f854d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:08:51.143191 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1226/2449] 10.222.0.23 () {58 vars in 967 bytes} [Mon May 9 23:08:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:08:51.155235 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac7b1e-2428-423c-a717-5281c6fdeef1 service nova] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:08:51.167030 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac7b1e-2428-423c-a717-5281c6fdeef1 service nova] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:08:51.171882 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d26788-349f-4e78-93de-23d9d64b3bd9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cb6c294ce1842b68a6bc73ac381ed6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:08:51.172529 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d26788-349f-4e78-93de-23d9d64b3bd9 service swift] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:08:51.172868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d26788-349f-4e78-93de-23d9d64b3bd9 service swift] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:08:51.173144 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d26788-349f-4e78-93de-23d9d64b3bd9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:08:51.181548 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d26788-349f-4e78-93de-23d9d64b3bd9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:08:51.183206 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d26788-349f-4e78-93de-23d9d64b3bd9 service swift] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:08:51.191061 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1224/2450] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 9 23:08:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 23:09:02.438759 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00824aee-91dc-4ca7-ada9-05fe7f854d08 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:02.458644 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00824aee-91dc-4ca7-ada9-05fe7f854d08 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:02.464410 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4da3e8-10bf-44d6-b2ac-5f0b05177fd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:02.464993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4da3e8-10bf-44d6-b2ac-5f0b05177fd5 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:02.465278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4da3e8-10bf-44d6-b2ac-5f0b05177fd5 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:02.465553 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4da3e8-10bf-44d6-b2ac-5f0b05177fd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:02.467967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7b4da3e8-10bf-44d6-b2ac-5f0b05177fd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:02.486479 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4da3e8-10bf-44d6-b2ac-5f0b05177fd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:02.488136 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4da3e8-10bf-44d6-b2ac-5f0b05177fd5 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:02.501447 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1227/2451] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:09:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:09:07.555115 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d26788-349f-4e78-93de-23d9d64b3bd9 service swift] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:07.564201 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d26788-349f-4e78-93de-23d9d64b3bd9 service swift] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:07.567789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87de2d7-2107-41b2-b18b-c71bf3b616f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:07.568412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87de2d7-2107-41b2-b18b-c71bf3b616f7 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:07.568693 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87de2d7-2107-41b2-b18b-c71bf3b616f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:07.568968 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87de2d7-2107-41b2-b18b-c71bf3b616f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:07.570724 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c87de2d7-2107-41b2-b18b-c71bf3b616f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:07.583467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87de2d7-2107-41b2-b18b-c71bf3b616f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:07.584960 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87de2d7-2107-41b2-b18b-c71bf3b616f7 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:07.594471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1225/2452] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:09:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:09:10.522371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4da3e8-10bf-44d6-b2ac-5f0b05177fd5 service nova] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:10.523804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7b4da3e8-10bf-44d6-b2ac-5f0b05177fd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:10.538770 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4da3e8-10bf-44d6-b2ac-5f0b05177fd5 service nova] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:10.543761 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe2023-01c9-45dd-8251-260e76a5ab6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:10.544708 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe2023-01c9-45dd-8251-260e76a5ab6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:10.545048 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe2023-01c9-45dd-8251-260e76a5ab6a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:10.545383 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe2023-01c9-45dd-8251-260e76a5ab6a admin admin] PATH_INFO: `/v3/users/84a45506c19c4cda8173c164a097f925` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:10.546693 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe2023-01c9-45dd-8251-260e76a5ab6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=84a45506c19c4cda8173c164a097f925)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:10.548795 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe2023-01c9-45dd-8251-260e76a5ab6a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:10.596204 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-94fe2023-01c9-45dd-8251-260e76a5ab6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84a45506c19c4cda8173c164a097f925'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:10.602791 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-94fe2023-01c9-45dd-8251-260e76a5ab6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84a45506c19c4cda8173c164a097f925'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:10.610145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-94fe2023-01c9-45dd-8251-260e76a5ab6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84a45506c19c4cda8173c164a097f925'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:10.610757 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-94fe2023-01c9-45dd-8251-260e76a5ab6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84a45506c19c4cda8173c164a097f925'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:10.621875 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-94fe2023-01c9-45dd-8251-260e76a5ab6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84a45506c19c4cda8173c164a097f925'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:10.627709 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1228/2453] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:09:10 2022] DELETE /identity/v3/users/84a45506c19c4cda8173c164a097f925 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:10.809665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87de2d7-2107-41b2-b18b-c71bf3b616f7 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:10.821547 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87de2d7-2107-41b2-b18b-c71bf3b616f7 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:10.878423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c31df-06c6-41ce-93b4-3396e92c5773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:10.879287 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c31df-06c6-41ce-93b4-3396e92c5773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:10.879561 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c31df-06c6-41ce-93b4-3396e92c5773 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:10.879828 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c31df-06c6-41ce-93b4-3396e92c5773 admin admin] PATH_INFO: `/v3/projects/b776734a879d4c0785b1d3728b79b766` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:10.880993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c31df-06c6-41ce-93b4-3396e92c5773 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b776734a879d4c0785b1d3728b79b766)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:10.882906 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c31df-06c6-41ce-93b4-3396e92c5773 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:10.927358 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-224c31df-06c6-41ce-93b4-3396e92c5773 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b776734a879d4c0785b1d3728b79b766', 'request_id': 'req-224c31df-06c6-41ce-93b4-3396e92c5773'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:10.933415 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-224c31df-06c6-41ce-93b4-3396e92c5773 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b776734a879d4c0785b1d3728b79b766', 'request_id': 'req-224c31df-06c6-41ce-93b4-3396e92c5773'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:10.939300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-224c31df-06c6-41ce-93b4-3396e92c5773 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b776734a879d4c0785b1d3728b79b766', 'request_id': 'req-224c31df-06c6-41ce-93b4-3396e92c5773'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:10.951124 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-224c31df-06c6-41ce-93b4-3396e92c5773 admin admin] The token cache is being invalidate because project b776734a879d4c0785b1d3728b79b766 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:09:10.951412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-224c31df-06c6-41ce-93b4-3396e92c5773 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:10.952030 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-224c31df-06c6-41ce-93b4-3396e92c5773 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:10.953608 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1226/2454] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:09:10 2022] DELETE /identity/v3/projects/b776734a879d4c0785b1d3728b79b766 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:10.959265 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fed98-4e06-481a-bc0f-2781f60c6f83 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:10.959857 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fed98-4e06-481a-bc0f-2781f60c6f83 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:10.960221 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fed98-4e06-481a-bc0f-2781f60c6f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:10.977669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-620fed98-4e06-481a-bc0f-2781f60c6f83 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:10.978900 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-620fed98-4e06-481a-bc0f-2781f60c6f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:11.036911 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1229/2455] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:11.045259 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a318ef6-9c80-47c9-875e-b0c05283b4b3 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:11.045552 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a318ef6-9c80-47c9-875e-b0c05283b4b3 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:11.045833 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a318ef6-9c80-47c9-875e-b0c05283b4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:11.064240 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-2a318ef6-9c80-47c9-875e-b0c05283b4b3 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:11.065356 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2a318ef6-9c80-47c9-875e-b0c05283b4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:11.073106 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1227/2456] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:11.077244 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fed98-4e06-481a-bc0f-2781f60c6f83 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:11.087433 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fed98-4e06-481a-bc0f-2781f60c6f83 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:11.092006 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4af254-f51f-4cec-ac6a-daea03e23744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:11.092651 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4af254-f51f-4cec-ac6a-daea03e23744 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:11.092934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4af254-f51f-4cec-ac6a-daea03e23744 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:11.093210 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4af254-f51f-4cec-ac6a-daea03e23744 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:11.093920 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4af254-f51f-4cec-ac6a-daea03e23744 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:09:11.094259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4af254-f51f-4cec-ac6a-daea03e23744 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:11.095867 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4af254-f51f-4cec-ac6a-daea03e23744 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:11.107253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1230/2457] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:09:11.111317 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a318ef6-9c80-47c9-875e-b0c05283b4b3 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:11.120328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a318ef6-9c80-47c9-875e-b0c05283b4b3 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:11.124930 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08811eee-8207-49b6-ad56-7720dbb0b544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:11.125591 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08811eee-8207-49b6-ad56-7720dbb0b544 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:11.125868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08811eee-8207-49b6-ad56-7720dbb0b544 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:11.126709 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08811eee-8207-49b6-ad56-7720dbb0b544 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:11.127395 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08811eee-8207-49b6-ad56-7720dbb0b544 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:11.129277 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08811eee-8207-49b6-ad56-7720dbb0b544 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:11.163575 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1228/2458] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:09:11 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:09:11.167614 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4af254-f51f-4cec-ac6a-daea03e23744 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:11.177934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4af254-f51f-4cec-ac6a-daea03e23744 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:11.230732 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e70a7f9-b035-4aa7-9d02-3db9ab26b911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:11.231650 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70a7f9-b035-4aa7-9d02-3db9ab26b911 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:11.231929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70a7f9-b035-4aa7-9d02-3db9ab26b911 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:11.232196 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70a7f9-b035-4aa7-9d02-3db9ab26b911 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:11.232857 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e70a7f9-b035-4aa7-9d02-3db9ab26b911 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:11.234769 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e70a7f9-b035-4aa7-9d02-3db9ab26b911 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:11.254012 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1231/2459] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:09:11 2022] POST /identity/v3/users => generated 283 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:09:11.258014 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08811eee-8207-49b6-ad56-7720dbb0b544 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:11.267795 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08811eee-8207-49b6-ad56-7720dbb0b544 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:11.272263 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feafdf3-aa83-485c-84d8-220bbd63a690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:11.272899 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feafdf3-aa83-485c-84d8-220bbd63a690 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:11.273174 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feafdf3-aa83-485c-84d8-220bbd63a690 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:11.273441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feafdf3-aa83-485c-84d8-220bbd63a690 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:11.274162 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feafdf3-aa83-485c-84d8-220bbd63a690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:11.275891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feafdf3-aa83-485c-84d8-220bbd63a690 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:11.285287 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1229/2460] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:09:11.289106 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e70a7f9-b035-4aa7-9d02-3db9ab26b911 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:11.299090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e70a7f9-b035-4aa7-9d02-3db9ab26b911 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:11.303913 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6e8a1-5661-4775-a5d2-1b7857853ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:11.304552 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6e8a1-5661-4775-a5d2-1b7857853ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:11.304831 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6e8a1-5661-4775-a5d2-1b7857853ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:11.305101 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6e8a1-5661-4775-a5d2-1b7857853ca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:11.305801 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6e8a1-5661-4775-a5d2-1b7857853ca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:11.307405 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6e8a1-5661-4775-a5d2-1b7857853ca8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:11.317405 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1232/2461] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:09:11.321433 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feafdf3-aa83-485c-84d8-220bbd63a690 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:11.331591 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feafdf3-aa83-485c-84d8-220bbd63a690 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:11.336201 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d64bcd-25db-4c7e-9dc8-c2e9d524d189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:11.337071 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d64bcd-25db-4c7e-9dc8-c2e9d524d189 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:11.337348 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d64bcd-25db-4c7e-9dc8-c2e9d524d189 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:11.337616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d64bcd-25db-4c7e-9dc8-c2e9d524d189 admin admin] PATH_INFO: `/v3/projects/88a00329813f4e6a94344368b9e9b342/users/1ae96a36f3594c39b345f5eee5dfdf47/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:11.352338 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d64bcd-25db-4c7e-9dc8-c2e9d524d189 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88a00329813f4e6a94344368b9e9b342, user_id=1ae96a36f3594c39b345f5eee5dfdf47, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:11.355070 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d64bcd-25db-4c7e-9dc8-c2e9d524d189 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:11.367720 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1230/2462] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:09:11 2022] PUT /identity/v3/projects/88a00329813f4e6a94344368b9e9b342/users/1ae96a36f3594c39b345f5eee5dfdf47/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:09:11.377761 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a009b-deb2-4ad2-a5d5-c9b0ae7a0292 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:11.378058 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a009b-deb2-4ad2-a5d5-c9b0ae7a0292 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:11.378393 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a009b-deb2-4ad2-a5d5-c9b0ae7a0292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:11.405024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-488a009b-deb2-4ad2-a5d5-c9b0ae7a0292 None None] MFA Rules not processed for user `1ae96a36f3594c39b345f5eee5dfdf47`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:11.406370 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-488a009b-deb2-4ad2-a5d5-c9b0ae7a0292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:11.468331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1233/2463] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:11 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:11.480728 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d64bcd-25db-4c7e-9dc8-c2e9d524d189 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:11.482701 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-16d64bcd-25db-4c7e-9dc8-c2e9d524d189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:11.539605 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d64bcd-25db-4c7e-9dc8-c2e9d524d189 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:11.543289 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f3aac9-0170-4ed5-9ef8-0c4024119b9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:11.543936 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3aac9-0170-4ed5-9ef8-0c4024119b9d service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:11.544216 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3aac9-0170-4ed5-9ef8-0c4024119b9d service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:11.544489 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3aac9-0170-4ed5-9ef8-0c4024119b9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:11.552947 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f3aac9-0170-4ed5-9ef8-0c4024119b9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:11.554407 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f3aac9-0170-4ed5-9ef8-0c4024119b9d service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:11.563428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1231/2464] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:09:11 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:09:19.807058 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a009b-deb2-4ad2-a5d5-c9b0ae7a0292 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:19.868436 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a009b-deb2-4ad2-a5d5-c9b0ae7a0292 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:19.873167 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ea4bf-2c44-4b20-8847-9d9f64083020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:19.874031 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ea4bf-2c44-4b20-8847-9d9f64083020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:19.874355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ea4bf-2c44-4b20-8847-9d9f64083020 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:19.874632 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ea4bf-2c44-4b20-8847-9d9f64083020 admin admin] PATH_INFO: `/v3/users/1ae96a36f3594c39b345f5eee5dfdf47` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:19.875859 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ea4bf-2c44-4b20-8847-9d9f64083020 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ae96a36f3594c39b345f5eee5dfdf47)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:19.878373 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ea4bf-2c44-4b20-8847-9d9f64083020 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:19.922134 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8f8ea4bf-2c44-4b20-8847-9d9f64083020 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ae96a36f3594c39b345f5eee5dfdf47'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:19.928643 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8f8ea4bf-2c44-4b20-8847-9d9f64083020 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ae96a36f3594c39b345f5eee5dfdf47'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:19.935559 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8f8ea4bf-2c44-4b20-8847-9d9f64083020 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ae96a36f3594c39b345f5eee5dfdf47'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:19.936168 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8f8ea4bf-2c44-4b20-8847-9d9f64083020 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ae96a36f3594c39b345f5eee5dfdf47'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:19.947861 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8f8ea4bf-2c44-4b20-8847-9d9f64083020 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ae96a36f3594c39b345f5eee5dfdf47'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:19.953489 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1234/2465] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:09:19 2022] DELETE /identity/v3/users/1ae96a36f3594c39b345f5eee5dfdf47 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:19.958632 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf042d-0d71-4686-afe3-6d4c06fab580 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:19.959111 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf042d-0d71-4686-afe3-6d4c06fab580 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:19.959234 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf042d-0d71-4686-afe3-6d4c06fab580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:19.977805 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-45bf042d-0d71-4686-afe3-6d4c06fab580 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:19.979041 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-45bf042d-0d71-4686-afe3-6d4c06fab580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:20.036102 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1232/2466] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:20.045599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ea4bf-2c44-4b20-8847-9d9f64083020 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:20.047039 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-8f8ea4bf-2c44-4b20-8847-9d9f64083020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:20.102783 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ea4bf-2c44-4b20-8847-9d9f64083020 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:20.106945 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953e557-3784-4603-8e60-00ddf5a0f983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:20.107591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953e557-3784-4603-8e60-00ddf5a0f983 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:20.107863 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953e557-3784-4603-8e60-00ddf5a0f983 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:20.108139 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953e557-3784-4603-8e60-00ddf5a0f983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:20.116992 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953e557-3784-4603-8e60-00ddf5a0f983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:20.118497 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953e557-3784-4603-8e60-00ddf5a0f983 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:20.128273 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1235/2467] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:09:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:09:20.383158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf042d-0d71-4686-afe3-6d4c06fab580 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:20.384607 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-45bf042d-0d71-4686-afe3-6d4c06fab580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:20.398272 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf042d-0d71-4686-afe3-6d4c06fab580 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:20.402910 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85b05a-c500-4219-8462-47625a7f719a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:20.403744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85b05a-c500-4219-8462-47625a7f719a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:20.404023 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85b05a-c500-4219-8462-47625a7f719a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:20.404295 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85b05a-c500-4219-8462-47625a7f719a admin admin] PATH_INFO: `/v3/projects/88a00329813f4e6a94344368b9e9b342` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:20.405977 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85b05a-c500-4219-8462-47625a7f719a admin admin] RBAC: Authorizing `identity:delete_project(project_id=88a00329813f4e6a94344368b9e9b342)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:20.408101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85b05a-c500-4219-8462-47625a7f719a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:20.453689 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ff85b05a-c500-4219-8462-47625a7f719a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88a00329813f4e6a94344368b9e9b342', 'request_id': 'req-ff85b05a-c500-4219-8462-47625a7f719a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:20.460066 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ff85b05a-c500-4219-8462-47625a7f719a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88a00329813f4e6a94344368b9e9b342', 'request_id': 'req-ff85b05a-c500-4219-8462-47625a7f719a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:20.465311 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ff85b05a-c500-4219-8462-47625a7f719a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88a00329813f4e6a94344368b9e9b342', 'request_id': 'req-ff85b05a-c500-4219-8462-47625a7f719a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:20.477381 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ff85b05a-c500-4219-8462-47625a7f719a admin admin] The token cache is being invalidate because project 88a00329813f4e6a94344368b9e9b342 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:09:20.477667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ff85b05a-c500-4219-8462-47625a7f719a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:20.478314 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ff85b05a-c500-4219-8462-47625a7f719a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:20.479840 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1233/2468] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:09:20 2022] DELETE /identity/v3/projects/88a00329813f4e6a94344368b9e9b342 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:20.485003 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f116e8-c28a-4990-a6e3-b4451538fb10 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:20.485294 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f116e8-c28a-4990-a6e3-b4451538fb10 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:20.485592 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f116e8-c28a-4990-a6e3-b4451538fb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:20.503657 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-66f116e8-c28a-4990-a6e3-b4451538fb10 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:20.504846 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-66f116e8-c28a-4990-a6e3-b4451538fb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:20.566904 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1236/2469] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:20.575784 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ad752-6fc6-4181-88d7-08c467bf22b2 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:20.576064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ad752-6fc6-4181-88d7-08c467bf22b2 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:20.576333 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ad752-6fc6-4181-88d7-08c467bf22b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:20.594010 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-7b8ad752-6fc6-4181-88d7-08c467bf22b2 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:20.595705 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7b8ad752-6fc6-4181-88d7-08c467bf22b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:20.603265 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1234/2470] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:09:20.607651 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f116e8-c28a-4990-a6e3-b4451538fb10 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:20.617586 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f116e8-c28a-4990-a6e3-b4451538fb10 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:20.622063 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc1fce-08db-4064-9c2d-e47888b7377e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:20.622725 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc1fce-08db-4064-9c2d-e47888b7377e admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:20.623000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc1fce-08db-4064-9c2d-e47888b7377e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:20.623274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc1fce-08db-4064-9c2d-e47888b7377e admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:20.623976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc1fce-08db-4064-9c2d-e47888b7377e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:09:20.624275 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc1fce-08db-4064-9c2d-e47888b7377e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:20.625848 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc1fce-08db-4064-9c2d-e47888b7377e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:20.637345 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1237/2471] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 23:09:20.641201 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ad752-6fc6-4181-88d7-08c467bf22b2 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:20.650290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ad752-6fc6-4181-88d7-08c467bf22b2 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:20.654838 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6193b777-d8bc-4c97-b83c-c240d0801145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:20.655469 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193b777-d8bc-4c97-b83c-c240d0801145 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:20.655743 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193b777-d8bc-4c97-b83c-c240d0801145 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:20.656010 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193b777-d8bc-4c97-b83c-c240d0801145 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:20.656684 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6193b777-d8bc-4c97-b83c-c240d0801145 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:20.659268 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6193b777-d8bc-4c97-b83c-c240d0801145 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:20.684260 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1235/2472] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:09:20 2022] POST /identity/v3/projects => generated 363 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:09:20.688464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc1fce-08db-4064-9c2d-e47888b7377e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:20.698304 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc1fce-08db-4064-9c2d-e47888b7377e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:20.757522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e7e53-8aca-4a3e-bcda-c8e16714ff25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:20.758214 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e7e53-8aca-4a3e-bcda-c8e16714ff25 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:20.758492 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e7e53-8aca-4a3e-bcda-c8e16714ff25 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:20.758764 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e7e53-8aca-4a3e-bcda-c8e16714ff25 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:20.759452 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e7e53-8aca-4a3e-bcda-c8e16714ff25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:20.761348 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e7e53-8aca-4a3e-bcda-c8e16714ff25 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:20.780538 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1238/2473] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:09:20 2022] POST /identity/v3/users => generated 282 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:09:20.785216 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6193b777-d8bc-4c97-b83c-c240d0801145 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:20.795369 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6193b777-d8bc-4c97-b83c-c240d0801145 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:20.800662 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe8e3f-ecc2-4896-97ab-db046ac21942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:20.801311 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe8e3f-ecc2-4896-97ab-db046ac21942 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:20.801591 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe8e3f-ecc2-4896-97ab-db046ac21942 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:20.801863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe8e3f-ecc2-4896-97ab-db046ac21942 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:20.802612 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe8e3f-ecc2-4896-97ab-db046ac21942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:20.804191 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe8e3f-ecc2-4896-97ab-db046ac21942 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:20.814226 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1236/2474] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:09:20.818172 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e7e53-8aca-4a3e-bcda-c8e16714ff25 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:20.827698 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e7e53-8aca-4a3e-bcda-c8e16714ff25 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:20.832233 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa55f9d-1f80-4463-91dd-9d9b3bddc881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:20.832870 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa55f9d-1f80-4463-91dd-9d9b3bddc881 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:20.833149 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa55f9d-1f80-4463-91dd-9d9b3bddc881 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:20.833423 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa55f9d-1f80-4463-91dd-9d9b3bddc881 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:20.834161 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa55f9d-1f80-4463-91dd-9d9b3bddc881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:20.835735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa55f9d-1f80-4463-91dd-9d9b3bddc881 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:20.846868 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1239/2475] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:09:20.850762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe8e3f-ecc2-4896-97ab-db046ac21942 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:20.860521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe8e3f-ecc2-4896-97ab-db046ac21942 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:20.865590 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd14ddb-c9a7-4edf-9efd-0cc489d9e4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:20.866489 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd14ddb-c9a7-4edf-9efd-0cc489d9e4ed admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:20.866767 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd14ddb-c9a7-4edf-9efd-0cc489d9e4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:20.867038 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd14ddb-c9a7-4edf-9efd-0cc489d9e4ed admin admin] PATH_INFO: `/v3/projects/89bfe4b4878d4c4a83d0b731b6126040/users/2f78069afef14778912586d97d8002e8/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:20.882596 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd14ddb-c9a7-4edf-9efd-0cc489d9e4ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=89bfe4b4878d4c4a83d0b731b6126040, user_id=2f78069afef14778912586d97d8002e8, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:20.885219 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd14ddb-c9a7-4edf-9efd-0cc489d9e4ed admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:20.898028 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1237/2476] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:09:20 2022] PUT /identity/v3/projects/89bfe4b4878d4c4a83d0b731b6126040/users/2f78069afef14778912586d97d8002e8/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:20.908928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599c05b-f77d-40a2-9f1a-a95a0c7143de None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:20.909581 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599c05b-f77d-40a2-9f1a-a95a0c7143de None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:20.909881 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599c05b-f77d-40a2-9f1a-a95a0c7143de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:20.936022 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-1599c05b-f77d-40a2-9f1a-a95a0c7143de None None] MFA Rules not processed for user `2f78069afef14778912586d97d8002e8`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:20.937208 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1599c05b-f77d-40a2-9f1a-a95a0c7143de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:20.998003 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1240/2477] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:20 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:21.002418 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd14ddb-c9a7-4edf-9efd-0cc489d9e4ed admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:21.060988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd14ddb-c9a7-4edf-9efd-0cc489d9e4ed admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:21.065590 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1669d7d-007d-4531-ac58-99d64a484aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:21.066267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1669d7d-007d-4531-ac58-99d64a484aad admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:21.066546 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1669d7d-007d-4531-ac58-99d64a484aad admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:21.066815 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1669d7d-007d-4531-ac58-99d64a484aad admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:21.067497 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1669d7d-007d-4531-ac58-99d64a484aad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:21.069441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1669d7d-007d-4531-ac58-99d64a484aad admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:21.093900 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1238/2478] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:09:21 2022] POST /identity/v3/projects => generated 365 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:09:21.098485 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1599c05b-f77d-40a2-9f1a-a95a0c7143de None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:21.109980 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1599c05b-f77d-40a2-9f1a-a95a0c7143de None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:21.163793 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85577748-35b6-4738-b03c-02483d3005c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:21.164463 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85577748-35b6-4738-b03c-02483d3005c3 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:21.164745 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85577748-35b6-4738-b03c-02483d3005c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:21.165042 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85577748-35b6-4738-b03c-02483d3005c3 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:21.165730 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85577748-35b6-4738-b03c-02483d3005c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:21.167658 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85577748-35b6-4738-b03c-02483d3005c3 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:21.186275 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1241/2479] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:09:21 2022] POST /identity/v3/users => generated 289 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:09:21.190288 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1669d7d-007d-4531-ac58-99d64a484aad admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:21.200376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1669d7d-007d-4531-ac58-99d64a484aad admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:21.205548 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf2f50-954d-4f32-a4a2-531e0f07b76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:21.206232 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf2f50-954d-4f32-a4a2-531e0f07b76b admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:21.206605 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf2f50-954d-4f32-a4a2-531e0f07b76b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:21.206911 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf2f50-954d-4f32-a4a2-531e0f07b76b admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:21.207623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdf2f50-954d-4f32-a4a2-531e0f07b76b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:21.209460 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdf2f50-954d-4f32-a4a2-531e0f07b76b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:21.219727 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1239/2480] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:09:21.224426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85577748-35b6-4738-b03c-02483d3005c3 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:21.234386 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85577748-35b6-4738-b03c-02483d3005c3 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:21.238921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5bb872-7ef7-4baa-ae51-215e20c948b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:21.239781 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bb872-7ef7-4baa-ae51-215e20c948b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:21.240059 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bb872-7ef7-4baa-ae51-215e20c948b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:21.240333 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bb872-7ef7-4baa-ae51-215e20c948b4 admin admin] PATH_INFO: `/v3/projects/25f0849ae3f348b89b214662ba0626dc/users/a91bfa748e0c4ccaa43b73e2b742faf9/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:21.254381 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5bb872-7ef7-4baa-ae51-215e20c948b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25f0849ae3f348b89b214662ba0626dc, user_id=a91bfa748e0c4ccaa43b73e2b742faf9, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:21.257010 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5bb872-7ef7-4baa-ae51-215e20c948b4 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:21.270670 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1242/2481] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:09:21 2022] PUT /identity/v3/projects/25f0849ae3f348b89b214662ba0626dc/users/a91bfa748e0c4ccaa43b73e2b742faf9/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:21.274588 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf2f50-954d-4f32-a4a2-531e0f07b76b admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:21.284836 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf2f50-954d-4f32-a4a2-531e0f07b76b admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:21.338179 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21246e-0743-42d1-b62e-f40053987b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:21.338772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21246e-0743-42d1-b62e-f40053987b2e admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:21.339048 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21246e-0743-42d1-b62e-f40053987b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:21.339320 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21246e-0743-42d1-b62e-f40053987b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:21.340023 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b21246e-0743-42d1-b62e-f40053987b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:21.341632 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b21246e-0743-42d1-b62e-f40053987b2e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:21.350923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1240/2482] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:09:21.354982 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5bb872-7ef7-4baa-ae51-215e20c948b4 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:21.365176 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5bb872-7ef7-4baa-ae51-215e20c948b4 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:21.369705 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd06f90-f2a1-4432-b4e3-aaa10e509fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:21.370607 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd06f90-f2a1-4432-b4e3-aaa10e509fab admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:21.370889 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd06f90-f2a1-4432-b4e3-aaa10e509fab admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:21.371164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd06f90-f2a1-4432-b4e3-aaa10e509fab admin admin] PATH_INFO: `/v3/projects/25f0849ae3f348b89b214662ba0626dc/users/a91bfa748e0c4ccaa43b73e2b742faf9/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:21.373096 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd06f90-f2a1-4432-b4e3-aaa10e509fab admin admin] RBAC: Authorizing `identity:create_grant(project_id=25f0849ae3f348b89b214662ba0626dc, user_id=a91bfa748e0c4ccaa43b73e2b742faf9, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:21.375748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd06f90-f2a1-4432-b4e3-aaa10e509fab admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:21.390513 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1243/2483] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:09:21 2022] PUT /identity/v3/projects/25f0849ae3f348b89b214662ba0626dc/users/a91bfa748e0c4ccaa43b73e2b742faf9/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:21.402983 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfeb66-9c67-448d-872f-5e5cb5c40879 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:21.403266 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfeb66-9c67-448d-872f-5e5cb5c40879 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:21.403534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfeb66-9c67-448d-872f-5e5cb5c40879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:21.429013 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-bdcfeb66-9c67-448d-872f-5e5cb5c40879 None None] MFA Rules not processed for user `a91bfa748e0c4ccaa43b73e2b742faf9`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:21.430242 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-bdcfeb66-9c67-448d-872f-5e5cb5c40879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:21.496456 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1241/2484] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:21 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:21.508362 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd06f90-f2a1-4432-b4e3-aaa10e509fab admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:21.510134 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-2bd06f90-f2a1-4432-b4e3-aaa10e509fab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:21.565747 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd06f90-f2a1-4432-b4e3-aaa10e509fab admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:21.569325 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b7639-3625-4886-bc95-5449dc3420b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:21.569950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b7639-3625-4886-bc95-5449dc3420b3 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:21.570261 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b7639-3625-4886-bc95-5449dc3420b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:21.570530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b7639-3625-4886-bc95-5449dc3420b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:21.580462 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b7639-3625-4886-bc95-5449dc3420b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:21.582061 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b7639-3625-4886-bc95-5449dc3420b3 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:21.591838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1244/2485] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:09:21 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:09:21.661822 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfeb66-9c67-448d-872f-5e5cb5c40879 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:21.671206 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfeb66-9c67-448d-872f-5e5cb5c40879 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:21.676895 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e223b-32f6-446e-a4a1-a132aeca9fde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:21.677533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e223b-32f6-446e-a4a1-a132aeca9fde service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:21.677807 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e223b-32f6-446e-a4a1-a132aeca9fde service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:21.678096 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e223b-32f6-446e-a4a1-a132aeca9fde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:21.739023 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e223b-32f6-446e-a4a1-a132aeca9fde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:21.740456 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e223b-32f6-446e-a4a1-a132aeca9fde service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:21.749201 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1242/2486] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:09:21 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:09:24.452638 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b7639-3625-4886-bc95-5449dc3420b3 service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:24.512246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b7639-3625-4886-bc95-5449dc3420b3 service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:24.516714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be5abf2-3777-4812-a701-245bdd13a94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:24.517564 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5abf2-3777-4812-a701-245bdd13a94b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:24.517843 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5abf2-3777-4812-a701-245bdd13a94b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:24.518140 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5abf2-3777-4812-a701-245bdd13a94b admin admin] PATH_INFO: `/v3/users/2f78069afef14778912586d97d8002e8` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:24.519974 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be5abf2-3777-4812-a701-245bdd13a94b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f78069afef14778912586d97d8002e8)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:24.521877 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be5abf2-3777-4812-a701-245bdd13a94b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:24.564149 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2be5abf2-3777-4812-a701-245bdd13a94b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f78069afef14778912586d97d8002e8'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:24.571136 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2be5abf2-3777-4812-a701-245bdd13a94b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f78069afef14778912586d97d8002e8'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:24.579201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2be5abf2-3777-4812-a701-245bdd13a94b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f78069afef14778912586d97d8002e8'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:24.579760 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2be5abf2-3777-4812-a701-245bdd13a94b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f78069afef14778912586d97d8002e8'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:24.591909 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2be5abf2-3777-4812-a701-245bdd13a94b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f78069afef14778912586d97d8002e8'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:24.597692 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1245/2487] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:09:24 2022] DELETE /identity/v3/users/2f78069afef14778912586d97d8002e8 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:24.602183 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1feebcf-ef62-414d-a958-ea44c6e90df6 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:24.602684 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1feebcf-ef62-414d-a958-ea44c6e90df6 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:24.602792 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1feebcf-ef62-414d-a958-ea44c6e90df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:24.620199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b1feebcf-ef62-414d-a958-ea44c6e90df6 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:24.621367 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b1feebcf-ef62-414d-a958-ea44c6e90df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:24.678462 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1243/2488] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:09:24.688161 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be5abf2-3777-4812-a701-245bdd13a94b admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:24.689533 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-2be5abf2-3777-4812-a701-245bdd13a94b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:24.752413 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be5abf2-3777-4812-a701-245bdd13a94b admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:24.756818 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe66a13-578f-4b2b-b2d0-9c017e1eae63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:24.757464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe66a13-578f-4b2b-b2d0-9c017e1eae63 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:24.757746 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe66a13-578f-4b2b-b2d0-9c017e1eae63 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:24.758021 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe66a13-578f-4b2b-b2d0-9c017e1eae63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:24.767233 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe66a13-578f-4b2b-b2d0-9c017e1eae63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:24.768780 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe66a13-578f-4b2b-b2d0-9c017e1eae63 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:24.779460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1246/2489] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:09:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:09:25.032353 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1feebcf-ef62-414d-a958-ea44c6e90df6 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:25.033766 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b1feebcf-ef62-414d-a958-ea44c6e90df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:25.048060 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1feebcf-ef62-414d-a958-ea44c6e90df6 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:25.053329 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9191da9-60b3-4643-857d-c74b5a41d3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:25.054267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9191da9-60b3-4643-857d-c74b5a41d3e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:25.054540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9191da9-60b3-4643-857d-c74b5a41d3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:25.054811 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9191da9-60b3-4643-857d-c74b5a41d3e9 admin admin] PATH_INFO: `/v3/projects/89bfe4b4878d4c4a83d0b731b6126040` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:25.055983 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9191da9-60b3-4643-857d-c74b5a41d3e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89bfe4b4878d4c4a83d0b731b6126040)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:25.057911 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9191da9-60b3-4643-857d-c74b5a41d3e9 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:25.108921 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f9191da9-60b3-4643-857d-c74b5a41d3e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89bfe4b4878d4c4a83d0b731b6126040', 'request_id': 'req-f9191da9-60b3-4643-857d-c74b5a41d3e9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:25.115932 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f9191da9-60b3-4643-857d-c74b5a41d3e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89bfe4b4878d4c4a83d0b731b6126040', 'request_id': 'req-f9191da9-60b3-4643-857d-c74b5a41d3e9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:25.121298 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f9191da9-60b3-4643-857d-c74b5a41d3e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89bfe4b4878d4c4a83d0b731b6126040', 'request_id': 'req-f9191da9-60b3-4643-857d-c74b5a41d3e9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:25.132999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f9191da9-60b3-4643-857d-c74b5a41d3e9 admin admin] The token cache is being invalidate because project 89bfe4b4878d4c4a83d0b731b6126040 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:09:25.133317 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f9191da9-60b3-4643-857d-c74b5a41d3e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:25.133942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f9191da9-60b3-4643-857d-c74b5a41d3e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:25.135508 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1244/2490] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:09:25 2022] DELETE /identity/v3/projects/89bfe4b4878d4c4a83d0b731b6126040 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:25.139499 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe66a13-578f-4b2b-b2d0-9c017e1eae63 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:25.148851 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe66a13-578f-4b2b-b2d0-9c017e1eae63 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:25.201263 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67977c-1e91-48f8-a126-9e3f49793709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:25.202103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67977c-1e91-48f8-a126-9e3f49793709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:25.202408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67977c-1e91-48f8-a126-9e3f49793709 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:25.202677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67977c-1e91-48f8-a126-9e3f49793709 admin admin] PATH_INFO: `/v3/users/a91bfa748e0c4ccaa43b73e2b742faf9` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:25.203823 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67977c-1e91-48f8-a126-9e3f49793709 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a91bfa748e0c4ccaa43b73e2b742faf9)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:25.205708 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67977c-1e91-48f8-a126-9e3f49793709 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:25.248939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9e67977c-1e91-48f8-a126-9e3f49793709 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a91bfa748e0c4ccaa43b73e2b742faf9'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:25.255353 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9e67977c-1e91-48f8-a126-9e3f49793709 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a91bfa748e0c4ccaa43b73e2b742faf9'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:25.261862 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9e67977c-1e91-48f8-a126-9e3f49793709 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a91bfa748e0c4ccaa43b73e2b742faf9'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:25.262565 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9e67977c-1e91-48f8-a126-9e3f49793709 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a91bfa748e0c4ccaa43b73e2b742faf9'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:25.273739 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9e67977c-1e91-48f8-a126-9e3f49793709 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a91bfa748e0c4ccaa43b73e2b742faf9'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:25.279424 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1247/2491] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:09:25 2022] DELETE /identity/v3/users/a91bfa748e0c4ccaa43b73e2b742faf9 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:25.562741 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9191da9-60b3-4643-857d-c74b5a41d3e9 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:25.564572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f9191da9-60b3-4643-857d-c74b5a41d3e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:25.578640 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9191da9-60b3-4643-857d-c74b5a41d3e9 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:25.644022 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1b53f3-3871-4401-9f94-619714fb3df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:25.645552 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b53f3-3871-4401-9f94-619714fb3df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:25.645892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b53f3-3871-4401-9f94-619714fb3df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:25.646281 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b53f3-3871-4401-9f94-619714fb3df2 admin admin] PATH_INFO: `/v3/projects/25f0849ae3f348b89b214662ba0626dc` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:25.647498 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1b53f3-3871-4401-9f94-619714fb3df2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25f0849ae3f348b89b214662ba0626dc)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:25.649412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1b53f3-3871-4401-9f94-619714fb3df2 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:25.700351 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8f1b53f3-3871-4401-9f94-619714fb3df2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25f0849ae3f348b89b214662ba0626dc', 'request_id': 'req-8f1b53f3-3871-4401-9f94-619714fb3df2'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:25.707612 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8f1b53f3-3871-4401-9f94-619714fb3df2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25f0849ae3f348b89b214662ba0626dc', 'request_id': 'req-8f1b53f3-3871-4401-9f94-619714fb3df2'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:25.713593 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8f1b53f3-3871-4401-9f94-619714fb3df2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25f0849ae3f348b89b214662ba0626dc', 'request_id': 'req-8f1b53f3-3871-4401-9f94-619714fb3df2'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:25.726086 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8f1b53f3-3871-4401-9f94-619714fb3df2 admin admin] The token cache is being invalidate because project 25f0849ae3f348b89b214662ba0626dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:09:25.726393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8f1b53f3-3871-4401-9f94-619714fb3df2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:25.730619 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8f1b53f3-3871-4401-9f94-619714fb3df2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:25.733125 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1245/2492] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:09:25 2022] DELETE /identity/v3/projects/25f0849ae3f348b89b214662ba0626dc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:25.737914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d8bee-2091-4535-b1a9-8caaf2e433e3 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:25.738454 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d8bee-2091-4535-b1a9-8caaf2e433e3 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:25.738561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d8bee-2091-4535-b1a9-8caaf2e433e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:25.756132 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-076d8bee-2091-4535-b1a9-8caaf2e433e3 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:25.757308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-076d8bee-2091-4535-b1a9-8caaf2e433e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:25.814098 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1248/2493] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:09:25.823528 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3062c3-b9b4-40e3-834d-f55a18bcfaf7 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:25.824042 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3062c3-b9b4-40e3-834d-f55a18bcfaf7 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:25.824184 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3062c3-b9b4-40e3-834d-f55a18bcfaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:25.843073 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ec3062c3-b9b4-40e3-834d-f55a18bcfaf7 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:25.844328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ec3062c3-b9b4-40e3-834d-f55a18bcfaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:25.851964 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1246/2494] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:09:25.856338 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d8bee-2091-4535-b1a9-8caaf2e433e3 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:25.865975 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d8bee-2091-4535-b1a9-8caaf2e433e3 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:25.870471 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e62df4-6bc7-414b-9f3a-678a5df41a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:25.871140 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e62df4-6bc7-414b-9f3a-678a5df41a7e admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:25.871416 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e62df4-6bc7-414b-9f3a-678a5df41a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:25.871683 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e62df4-6bc7-414b-9f3a-678a5df41a7e admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:25.872389 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e62df4-6bc7-414b-9f3a-678a5df41a7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:09:25.872688 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e62df4-6bc7-414b-9f3a-678a5df41a7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:25.874286 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e62df4-6bc7-414b-9f3a-678a5df41a7e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:25.885863 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1249/2495] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 23:09:25.889795 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3062c3-b9b4-40e3-834d-f55a18bcfaf7 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:25.899181 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3062c3-b9b4-40e3-834d-f55a18bcfaf7 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:25.903647 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f31ef9-1c39-4eb2-b1e3-462dd74c47f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:25.904272 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f31ef9-1c39-4eb2-b1e3-462dd74c47f1 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:25.904545 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f31ef9-1c39-4eb2-b1e3-462dd74c47f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:25.904809 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f31ef9-1c39-4eb2-b1e3-462dd74c47f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:25.905479 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f31ef9-1c39-4eb2-b1e3-462dd74c47f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:25.907373 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f31ef9-1c39-4eb2-b1e3-462dd74c47f1 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:25.930916 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1247/2496] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:09:25 2022] POST /identity/v3/projects => generated 369 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:09:25.934879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e62df4-6bc7-414b-9f3a-678a5df41a7e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:25.945147 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e62df4-6bc7-414b-9f3a-678a5df41a7e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:25.998211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4ebc8-f2c5-4b76-90d7-9462658e2912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:25.998864 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4ebc8-f2c5-4b76-90d7-9462658e2912 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:25.999144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4ebc8-f2c5-4b76-90d7-9462658e2912 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:25.999417 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4ebc8-f2c5-4b76-90d7-9462658e2912 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:26.000096 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee4ebc8-f2c5-4b76-90d7-9462658e2912 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:26.001996 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee4ebc8-f2c5-4b76-90d7-9462658e2912 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:26.023761 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1250/2497] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:09:25 2022] POST /identity/v3/users => generated 285 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:09:26.027845 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f31ef9-1c39-4eb2-b1e3-462dd74c47f1 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:26.037683 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f31ef9-1c39-4eb2-b1e3-462dd74c47f1 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:26.042168 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3a627-67dd-41f2-a04e-da6383ea920a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:26.042812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3a627-67dd-41f2-a04e-da6383ea920a admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:26.043087 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3a627-67dd-41f2-a04e-da6383ea920a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:26.043360 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3a627-67dd-41f2-a04e-da6383ea920a admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:26.044065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a3a627-67dd-41f2-a04e-da6383ea920a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:26.045638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a3a627-67dd-41f2-a04e-da6383ea920a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:26.055468 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1248/2498] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:09:26.059240 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4ebc8-f2c5-4b76-90d7-9462658e2912 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:26.069579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4ebc8-f2c5-4b76-90d7-9462658e2912 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:26.074086 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afe975-3545-4cfc-ab76-a7f1252e84a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:26.074755 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afe975-3545-4cfc-ab76-a7f1252e84a8 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:26.075033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afe975-3545-4cfc-ab76-a7f1252e84a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:26.075305 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afe975-3545-4cfc-ab76-a7f1252e84a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:26.076016 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4afe975-3545-4cfc-ab76-a7f1252e84a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:26.077588 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4afe975-3545-4cfc-ab76-a7f1252e84a8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:26.089674 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1251/2499] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:09:26.093655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3a627-67dd-41f2-a04e-da6383ea920a admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:26.103631 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3a627-67dd-41f2-a04e-da6383ea920a admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:26.112896 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca2beb-57d0-43ba-99a7-ffec42629d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:26.114338 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca2beb-57d0-43ba-99a7-ffec42629d27 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:26.114620 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca2beb-57d0-43ba-99a7-ffec42629d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:26.114896 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca2beb-57d0-43ba-99a7-ffec42629d27 admin admin] PATH_INFO: `/v3/projects/1f4b07fa084043c28c69eb46ae764a80/users/da5ab830fcab4423ba74dfceb79aa76d/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:26.132473 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ca2beb-57d0-43ba-99a7-ffec42629d27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f4b07fa084043c28c69eb46ae764a80, user_id=da5ab830fcab4423ba74dfceb79aa76d, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:26.135204 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ca2beb-57d0-43ba-99a7-ffec42629d27 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:26.147420 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1249/2500] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:09:26 2022] PUT /identity/v3/projects/1f4b07fa084043c28c69eb46ae764a80/users/da5ab830fcab4423ba74dfceb79aa76d/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:09:26.157637 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f923f0-f456-41c8-a8de-f0d8413ce6de None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:26.157937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f923f0-f456-41c8-a8de-f0d8413ce6de None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:26.158246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f923f0-f456-41c8-a8de-f0d8413ce6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:26.183502 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-26f923f0-f456-41c8-a8de-f0d8413ce6de None None] MFA Rules not processed for user `da5ab830fcab4423ba74dfceb79aa76d`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:26.184692 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-26f923f0-f456-41c8-a8de-f0d8413ce6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:26.245565 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1252/2501] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:26.249875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca2beb-57d0-43ba-99a7-ffec42629d27 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:26.308084 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca2beb-57d0-43ba-99a7-ffec42629d27 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:26.312650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d394af-5022-4c9d-837c-5e39ac446264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:26.313289 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d394af-5022-4c9d-837c-5e39ac446264 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:26.313565 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d394af-5022-4c9d-837c-5e39ac446264 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:26.313837 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d394af-5022-4c9d-837c-5e39ac446264 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:26.314565 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d394af-5022-4c9d-837c-5e39ac446264 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:26.317042 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d394af-5022-4c9d-837c-5e39ac446264 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:26.340779 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1250/2502] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:09:26 2022] POST /identity/v3/projects => generated 369 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:09:26.344755 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f923f0-f456-41c8-a8de-f0d8413ce6de None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:26.355276 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f923f0-f456-41c8-a8de-f0d8413ce6de None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:26.408680 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e0222-0385-4682-b44e-70d355f66d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:26.409263 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e0222-0385-4682-b44e-70d355f66d9c admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:26.409543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e0222-0385-4682-b44e-70d355f66d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:26.409820 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e0222-0385-4682-b44e-70d355f66d9c admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:26.410530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e0222-0385-4682-b44e-70d355f66d9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:26.412422 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e0222-0385-4682-b44e-70d355f66d9c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:26.431800 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1253/2503] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:09:26 2022] POST /identity/v3/users => generated 291 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:09:26.435669 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d394af-5022-4c9d-837c-5e39ac446264 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:26.446264 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d394af-5022-4c9d-837c-5e39ac446264 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:26.450802 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692c065-22c5-463e-989f-156f7d98679f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:26.451448 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692c065-22c5-463e-989f-156f7d98679f admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:26.451728 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692c065-22c5-463e-989f-156f7d98679f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:26.452006 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692c065-22c5-463e-989f-156f7d98679f admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:26.452747 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692c065-22c5-463e-989f-156f7d98679f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:26.454372 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692c065-22c5-463e-989f-156f7d98679f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:26.464387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1251/2504] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:09:26.468315 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e0222-0385-4682-b44e-70d355f66d9c admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:26.478653 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e0222-0385-4682-b44e-70d355f66d9c admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:26.483525 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeffeb6-1478-4ab5-8f7d-d12bf68500f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:26.484382 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeffeb6-1478-4ab5-8f7d-d12bf68500f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:26.484662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeffeb6-1478-4ab5-8f7d-d12bf68500f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:26.484941 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeffeb6-1478-4ab5-8f7d-d12bf68500f9 admin admin] PATH_INFO: `/v3/projects/7394e761be834bfea1091820b5889813/users/450e20e33f124ff98cdf78abf152c5ae/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:26.499624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeffeb6-1478-4ab5-8f7d-d12bf68500f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7394e761be834bfea1091820b5889813, user_id=450e20e33f124ff98cdf78abf152c5ae, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:26.502298 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeffeb6-1478-4ab5-8f7d-d12bf68500f9 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:26.514686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1254/2505] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:09:26 2022] PUT /identity/v3/projects/7394e761be834bfea1091820b5889813/users/450e20e33f124ff98cdf78abf152c5ae/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:26.518648 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692c065-22c5-463e-989f-156f7d98679f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:26.528779 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692c065-22c5-463e-989f-156f7d98679f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:26.583020 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2467c1e-e034-42ed-9600-4187b2ddc8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:26.583682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2467c1e-e034-42ed-9600-4187b2ddc8dd admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:26.583966 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2467c1e-e034-42ed-9600-4187b2ddc8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:26.584250 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2467c1e-e034-42ed-9600-4187b2ddc8dd admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:26.584975 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2467c1e-e034-42ed-9600-4187b2ddc8dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:26.586594 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2467c1e-e034-42ed-9600-4187b2ddc8dd admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:26.596249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1252/2506] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:09:26.600757 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeffeb6-1478-4ab5-8f7d-d12bf68500f9 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:26.611081 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeffeb6-1478-4ab5-8f7d-d12bf68500f9 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:26.616409 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ffb25-63da-4d8a-b237-5c603e628278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:26.617277 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ffb25-63da-4d8a-b237-5c603e628278 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:26.617559 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ffb25-63da-4d8a-b237-5c603e628278 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:26.617834 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ffb25-63da-4d8a-b237-5c603e628278 admin admin] PATH_INFO: `/v3/projects/7394e761be834bfea1091820b5889813/users/450e20e33f124ff98cdf78abf152c5ae/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:26.619817 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ffb25-63da-4d8a-b237-5c603e628278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7394e761be834bfea1091820b5889813, user_id=450e20e33f124ff98cdf78abf152c5ae, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:26.622521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ffb25-63da-4d8a-b237-5c603e628278 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:26.632736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1255/2507] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:09:26 2022] PUT /identity/v3/projects/7394e761be834bfea1091820b5889813/users/450e20e33f124ff98cdf78abf152c5ae/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:26.643715 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea1408-7a3e-4e4b-b0c4-9243b5bf8df4 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:26.644008 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea1408-7a3e-4e4b-b0c4-9243b5bf8df4 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:26.644291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea1408-7a3e-4e4b-b0c4-9243b5bf8df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:26.671262 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a4ea1408-7a3e-4e4b-b0c4-9243b5bf8df4 None None] MFA Rules not processed for user `450e20e33f124ff98cdf78abf152c5ae`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:26.672365 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a4ea1408-7a3e-4e4b-b0c4-9243b5bf8df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:26.735770 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1253/2508] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:26 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:09:26.747274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ffb25-63da-4d8a-b237-5c603e628278 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:26.749073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d76ffb25-63da-4d8a-b237-5c603e628278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:26.814386 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ffb25-63da-4d8a-b237-5c603e628278 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:26.818773 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6018f10-7d47-47a6-beb7-baabf9086511 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:26.819563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6018f10-7d47-47a6-beb7-baabf9086511 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:26.819906 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6018f10-7d47-47a6-beb7-baabf9086511 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:26.820289 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6018f10-7d47-47a6-beb7-baabf9086511 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:26.846688 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6018f10-7d47-47a6-beb7-baabf9086511 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:26.849484 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6018f10-7d47-47a6-beb7-baabf9086511 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:26.860818 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1256/2509] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:09:26 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:09:26.909227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ea1408-7a3e-4e4b-b0c4-9243b5bf8df4 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:26.911568 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a4ea1408-7a3e-4e4b-b0c4-9243b5bf8df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:26.924009 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ea1408-7a3e-4e4b-b0c4-9243b5bf8df4 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:26.927630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca0d41-b2ed-40ed-8f37-190fce9f3acf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:26.928269 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca0d41-b2ed-40ed-8f37-190fce9f3acf service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:26.928544 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca0d41-b2ed-40ed-8f37-190fce9f3acf service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:26.928811 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca0d41-b2ed-40ed-8f37-190fce9f3acf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:26.983654 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca0d41-b2ed-40ed-8f37-190fce9f3acf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:26.985073 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca0d41-b2ed-40ed-8f37-190fce9f3acf service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:26.993775 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1254/2510] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:09:26 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:09:30.290549 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6018f10-7d47-47a6-beb7-baabf9086511 service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:30.350738 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6018f10-7d47-47a6-beb7-baabf9086511 service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:30.355219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1378e7-7e5d-482d-aab7-affc7ce06b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:30.356315 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1378e7-7e5d-482d-aab7-affc7ce06b4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:30.356591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1378e7-7e5d-482d-aab7-affc7ce06b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:30.356861 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1378e7-7e5d-482d-aab7-affc7ce06b4c admin admin] PATH_INFO: `/v3/users/da5ab830fcab4423ba74dfceb79aa76d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:30.358024 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1378e7-7e5d-482d-aab7-affc7ce06b4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=da5ab830fcab4423ba74dfceb79aa76d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:30.359959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1378e7-7e5d-482d-aab7-affc7ce06b4c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:30.403365 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ae1378e7-7e5d-482d-aab7-affc7ce06b4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da5ab830fcab4423ba74dfceb79aa76d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:30.410458 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ae1378e7-7e5d-482d-aab7-affc7ce06b4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da5ab830fcab4423ba74dfceb79aa76d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:30.417184 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ae1378e7-7e5d-482d-aab7-affc7ce06b4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da5ab830fcab4423ba74dfceb79aa76d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:30.417796 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ae1378e7-7e5d-482d-aab7-affc7ce06b4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da5ab830fcab4423ba74dfceb79aa76d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:30.429960 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ae1378e7-7e5d-482d-aab7-affc7ce06b4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da5ab830fcab4423ba74dfceb79aa76d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:30.435596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1257/2511] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:09:30 2022] DELETE /identity/v3/users/da5ab830fcab4423ba74dfceb79aa76d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:30.440067 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277e605-abf3-4af1-8c4c-37b184d41edd None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:30.440570 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277e605-abf3-4af1-8c4c-37b184d41edd None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:30.440721 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277e605-abf3-4af1-8c4c-37b184d41edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:30.457378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-2277e605-abf3-4af1-8c4c-37b184d41edd None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:30.458600 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2277e605-abf3-4af1-8c4c-37b184d41edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:30.515054 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1255/2512] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:30.525092 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1378e7-7e5d-482d-aab7-affc7ce06b4c admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:30.526858 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ae1378e7-7e5d-482d-aab7-affc7ce06b4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:30.582704 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1378e7-7e5d-482d-aab7-affc7ce06b4c admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:30.586322 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c5026-f63b-4b84-876c-ed2e6201dbd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:30.586954 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c5026-f63b-4b84-876c-ed2e6201dbd8 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:30.587229 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c5026-f63b-4b84-876c-ed2e6201dbd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:30.587499 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c5026-f63b-4b84-876c-ed2e6201dbd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:30.596191 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c5026-f63b-4b84-876c-ed2e6201dbd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:30.598240 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c5026-f63b-4b84-876c-ed2e6201dbd8 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:30.607774 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1258/2513] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:09:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:09:30.865593 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277e605-abf3-4af1-8c4c-37b184d41edd None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:30.870027 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2277e605-abf3-4af1-8c4c-37b184d41edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:30.885203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277e605-abf3-4af1-8c4c-37b184d41edd None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:30.890608 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed96ee4-9546-48ea-9bb3-e060a1c360e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:30.891956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed96ee4-9546-48ea-9bb3-e060a1c360e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:30.892310 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed96ee4-9546-48ea-9bb3-e060a1c360e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:30.893193 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed96ee4-9546-48ea-9bb3-e060a1c360e4 admin admin] PATH_INFO: `/v3/projects/1f4b07fa084043c28c69eb46ae764a80` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:30.895836 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed96ee4-9546-48ea-9bb3-e060a1c360e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f4b07fa084043c28c69eb46ae764a80)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:30.902296 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed96ee4-9546-48ea-9bb3-e060a1c360e4 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:30.949879 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-6ed96ee4-9546-48ea-9bb3-e060a1c360e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f4b07fa084043c28c69eb46ae764a80', 'request_id': 'req-6ed96ee4-9546-48ea-9bb3-e060a1c360e4'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:30.955970 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-6ed96ee4-9546-48ea-9bb3-e060a1c360e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f4b07fa084043c28c69eb46ae764a80', 'request_id': 'req-6ed96ee4-9546-48ea-9bb3-e060a1c360e4'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:30.961659 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-6ed96ee4-9546-48ea-9bb3-e060a1c360e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f4b07fa084043c28c69eb46ae764a80', 'request_id': 'req-6ed96ee4-9546-48ea-9bb3-e060a1c360e4'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:30.975842 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-6ed96ee4-9546-48ea-9bb3-e060a1c360e4 admin admin] The token cache is being invalidate because project 1f4b07fa084043c28c69eb46ae764a80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:09:30.976220 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-6ed96ee4-9546-48ea-9bb3-e060a1c360e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:30.976882 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-6ed96ee4-9546-48ea-9bb3-e060a1c360e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:30.978480 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1256/2514] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:09:30 2022] DELETE /identity/v3/projects/1f4b07fa084043c28c69eb46ae764a80 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:30.983076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c5026-f63b-4b84-876c-ed2e6201dbd8 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:30.993327 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c5026-f63b-4b84-876c-ed2e6201dbd8 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:31.056499 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758430d9-5974-4ef9-ac77-ecc0f3d1958a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:31.057360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758430d9-5974-4ef9-ac77-ecc0f3d1958a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:31.057642 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758430d9-5974-4ef9-ac77-ecc0f3d1958a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:31.057918 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758430d9-5974-4ef9-ac77-ecc0f3d1958a admin admin] PATH_INFO: `/v3/users/450e20e33f124ff98cdf78abf152c5ae` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:31.059117 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758430d9-5974-4ef9-ac77-ecc0f3d1958a admin admin] RBAC: Authorizing `identity:delete_user(user_id=450e20e33f124ff98cdf78abf152c5ae)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:31.061033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758430d9-5974-4ef9-ac77-ecc0f3d1958a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:31.104511 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-758430d9-5974-4ef9-ac77-ecc0f3d1958a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '450e20e33f124ff98cdf78abf152c5ae'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:31.111492 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-758430d9-5974-4ef9-ac77-ecc0f3d1958a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '450e20e33f124ff98cdf78abf152c5ae'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:31.118772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-758430d9-5974-4ef9-ac77-ecc0f3d1958a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '450e20e33f124ff98cdf78abf152c5ae'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:31.119419 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-758430d9-5974-4ef9-ac77-ecc0f3d1958a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '450e20e33f124ff98cdf78abf152c5ae'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:31.131437 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-758430d9-5974-4ef9-ac77-ecc0f3d1958a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '450e20e33f124ff98cdf78abf152c5ae'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:31.137127 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1259/2515] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:09:30 2022] DELETE /identity/v3/users/450e20e33f124ff98cdf78abf152c5ae => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:31.395488 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed96ee4-9546-48ea-9bb3-e060a1c360e4 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:31.396894 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-6ed96ee4-9546-48ea-9bb3-e060a1c360e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:31.411426 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed96ee4-9546-48ea-9bb3-e060a1c360e4 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:31.466406 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb298eb-8bab-4234-b08b-1d13c0536bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:31.467322 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb298eb-8bab-4234-b08b-1d13c0536bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:31.467539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb298eb-8bab-4234-b08b-1d13c0536bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:31.467810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb298eb-8bab-4234-b08b-1d13c0536bcc admin admin] PATH_INFO: `/v3/projects/7394e761be834bfea1091820b5889813` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:31.468969 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb298eb-8bab-4234-b08b-1d13c0536bcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7394e761be834bfea1091820b5889813)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:31.470951 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb298eb-8bab-4234-b08b-1d13c0536bcc admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:31.515035 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8bb298eb-8bab-4234-b08b-1d13c0536bcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7394e761be834bfea1091820b5889813', 'request_id': 'req-8bb298eb-8bab-4234-b08b-1d13c0536bcc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:31.520947 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8bb298eb-8bab-4234-b08b-1d13c0536bcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7394e761be834bfea1091820b5889813', 'request_id': 'req-8bb298eb-8bab-4234-b08b-1d13c0536bcc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:31.526070 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8bb298eb-8bab-4234-b08b-1d13c0536bcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7394e761be834bfea1091820b5889813', 'request_id': 'req-8bb298eb-8bab-4234-b08b-1d13c0536bcc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:31.537191 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8bb298eb-8bab-4234-b08b-1d13c0536bcc admin admin] The token cache is being invalidate because project 7394e761be834bfea1091820b5889813 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:09:31.537473 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8bb298eb-8bab-4234-b08b-1d13c0536bcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:31.538074 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8bb298eb-8bab-4234-b08b-1d13c0536bcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:31.539604 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1257/2516] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:09:31 2022] DELETE /identity/v3/projects/7394e761be834bfea1091820b5889813 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:31.544566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077100a1-deff-4d12-b172-f7f95b9c9485 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:31.545038 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077100a1-deff-4d12-b172-f7f95b9c9485 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:31.545136 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077100a1-deff-4d12-b172-f7f95b9c9485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:31.562954 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-077100a1-deff-4d12-b172-f7f95b9c9485 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:31.564127 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-077100a1-deff-4d12-b172-f7f95b9c9485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:31.621282 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1260/2517] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:31.629897 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e557c-9ef6-4d77-b2b0-440b0369b72a None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:31.630227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e557c-9ef6-4d77-b2b0-440b0369b72a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:31.630508 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e557c-9ef6-4d77-b2b0-440b0369b72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:31.649355 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-305e557c-9ef6-4d77-b2b0-440b0369b72a None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:31.650580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-305e557c-9ef6-4d77-b2b0-440b0369b72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:31.658487 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1258/2518] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:31.662801 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077100a1-deff-4d12-b172-f7f95b9c9485 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:31.672194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077100a1-deff-4d12-b172-f7f95b9c9485 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:31.677188 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152611e9-7038-46ab-8869-864052fdd24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:31.677825 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152611e9-7038-46ab-8869-864052fdd24a admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:31.678100 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152611e9-7038-46ab-8869-864052fdd24a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:31.678400 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152611e9-7038-46ab-8869-864052fdd24a admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:31.679110 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152611e9-7038-46ab-8869-864052fdd24a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:09:31.679410 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152611e9-7038-46ab-8869-864052fdd24a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:31.680975 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152611e9-7038-46ab-8869-864052fdd24a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:31.692634 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1261/2519] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:09:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 23:09:31.696842 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e557c-9ef6-4d77-b2b0-440b0369b72a None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:31.705799 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e557c-9ef6-4d77-b2b0-440b0369b72a None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:31.710434 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38da6e0-f7c3-4fa0-b9a7-2f018b9e6c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:31.711069 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38da6e0-f7c3-4fa0-b9a7-2f018b9e6c40 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:31.711343 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38da6e0-f7c3-4fa0-b9a7-2f018b9e6c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:31.711638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38da6e0-f7c3-4fa0-b9a7-2f018b9e6c40 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:31.712317 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38da6e0-f7c3-4fa0-b9a7-2f018b9e6c40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:31.714222 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38da6e0-f7c3-4fa0-b9a7-2f018b9e6c40 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:31.738902 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1259/2520] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:09:31 2022] POST /identity/v3/projects => generated 375 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:09:31.743455 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152611e9-7038-46ab-8869-864052fdd24a admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:31.753125 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152611e9-7038-46ab-8869-864052fdd24a admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:31.807688 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdde347-f451-45b9-a59e-93d39013018d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:31.808332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdde347-f451-45b9-a59e-93d39013018d admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:31.808611 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdde347-f451-45b9-a59e-93d39013018d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:31.808882 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdde347-f451-45b9-a59e-93d39013018d admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:31.809585 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdde347-f451-45b9-a59e-93d39013018d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:31.811496 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdde347-f451-45b9-a59e-93d39013018d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:31.830502 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1262/2521] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:09:31 2022] POST /identity/v3/users => generated 288 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:09:31.834686 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38da6e0-f7c3-4fa0-b9a7-2f018b9e6c40 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:31.846240 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38da6e0-f7c3-4fa0-b9a7-2f018b9e6c40 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:31.851426 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38eaf3-f5b2-40f7-a5ad-0f602ffa6644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:31.852070 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38eaf3-f5b2-40f7-a5ad-0f602ffa6644 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:31.852344 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38eaf3-f5b2-40f7-a5ad-0f602ffa6644 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:31.852614 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38eaf3-f5b2-40f7-a5ad-0f602ffa6644 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:31.853323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38eaf3-f5b2-40f7-a5ad-0f602ffa6644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:31.854938 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38eaf3-f5b2-40f7-a5ad-0f602ffa6644 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:31.865688 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1260/2522] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:09:31.869662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdde347-f451-45b9-a59e-93d39013018d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:31.879397 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdde347-f451-45b9-a59e-93d39013018d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:31.883950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0308babe-48e4-412b-99e8-66b79a930a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:31.884591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308babe-48e4-412b-99e8-66b79a930a99 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:31.884873 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308babe-48e4-412b-99e8-66b79a930a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:31.885177 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308babe-48e4-412b-99e8-66b79a930a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:31.885895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0308babe-48e4-412b-99e8-66b79a930a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:31.887603 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0308babe-48e4-412b-99e8-66b79a930a99 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:31.897214 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1263/2523] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:09:31.901171 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38eaf3-f5b2-40f7-a5ad-0f602ffa6644 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:31.911663 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38eaf3-f5b2-40f7-a5ad-0f602ffa6644 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:31.916217 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150e0f6-9e11-4cc8-9d27-eded05fda40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:31.917162 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150e0f6-9e11-4cc8-9d27-eded05fda40e admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:31.917474 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150e0f6-9e11-4cc8-9d27-eded05fda40e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:31.917832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150e0f6-9e11-4cc8-9d27-eded05fda40e admin admin] PATH_INFO: `/v3/projects/ea4b83efa8ad41879ef0d18492aaecbf/users/ef5b51a6fdaa4a6286e1e9f007dac776/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:31.932715 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6150e0f6-9e11-4cc8-9d27-eded05fda40e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea4b83efa8ad41879ef0d18492aaecbf, user_id=ef5b51a6fdaa4a6286e1e9f007dac776, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:31.935384 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6150e0f6-9e11-4cc8-9d27-eded05fda40e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:31.947575 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1261/2524] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:09:31 2022] PUT /identity/v3/projects/ea4b83efa8ad41879ef0d18492aaecbf/users/ef5b51a6fdaa4a6286e1e9f007dac776/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:31.958033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874be6e-8333-4b83-8252-c79764f87508 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:31.958352 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874be6e-8333-4b83-8252-c79764f87508 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:31.958635 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874be6e-8333-4b83-8252-c79764f87508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:31.984792 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e874be6e-8333-4b83-8252-c79764f87508 None None] MFA Rules not processed for user `ef5b51a6fdaa4a6286e1e9f007dac776`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:31.985983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e874be6e-8333-4b83-8252-c79764f87508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:32.050994 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1264/2525] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:32.062502 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150e0f6-9e11-4cc8-9d27-eded05fda40e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:32.064240 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-6150e0f6-9e11-4cc8-9d27-eded05fda40e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:32.126519 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150e0f6-9e11-4cc8-9d27-eded05fda40e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:32.130733 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ed304-61eb-4594-9669-2787a582869f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81caf0558f2449cbbc8f4cd9446f363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:32.131365 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ed304-61eb-4594-9669-2787a582869f service glance] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:32.131638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ed304-61eb-4594-9669-2787a582869f service glance] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:32.131905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ed304-61eb-4594-9669-2787a582869f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:32.145042 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ed304-61eb-4594-9669-2787a582869f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:32.146187 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ed304-61eb-4594-9669-2787a582869f service glance] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:32.151463 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874be6e-8333-4b83-8252-c79764f87508 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:32.152922 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e874be6e-8333-4b83-8252-c79764f87508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:32.155435 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1262/2526] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:09:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:09:32.209432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874be6e-8333-4b83-8252-c79764f87508 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:32.213321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4d856-18d2-4337-b487-5bd5330d32ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:32.214093 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4d856-18d2-4337-b487-5bd5330d32ef service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:32.214411 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4d856-18d2-4337-b487-5bd5330d32ef service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:32.214691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4d856-18d2-4337-b487-5bd5330d32ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:32.216501 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b9b4d856-18d2-4337-b487-5bd5330d32ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:32.279353 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4d856-18d2-4337-b487-5bd5330d32ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:32.280821 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4d856-18d2-4337-b487-5bd5330d32ef service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:32.290431 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1265/2527] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:09:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 23:09:35.560930 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ed304-61eb-4594-9669-2787a582869f service glance] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:35.562430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-256ed304-61eb-4594-9669-2787a582869f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:35.578481 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ed304-61eb-4594-9669-2787a582869f service glance] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:35.583905 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840baff2-bf1d-4695-a96b-9d137f9a966b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:35.584777 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840baff2-bf1d-4695-a96b-9d137f9a966b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:35.585053 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840baff2-bf1d-4695-a96b-9d137f9a966b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:35.585329 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840baff2-bf1d-4695-a96b-9d137f9a966b admin admin] PATH_INFO: `/v3/users/c950754974d748519118d974492f15ce` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:35.586554 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840baff2-bf1d-4695-a96b-9d137f9a966b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c950754974d748519118d974492f15ce)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:35.588552 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840baff2-bf1d-4695-a96b-9d137f9a966b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:35.639644 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-840baff2-bf1d-4695-a96b-9d137f9a966b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c950754974d748519118d974492f15ce'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:35.647066 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-840baff2-bf1d-4695-a96b-9d137f9a966b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c950754974d748519118d974492f15ce'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:35.647723 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-840baff2-bf1d-4695-a96b-9d137f9a966b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c950754974d748519118d974492f15ce'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:35.654715 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-840baff2-bf1d-4695-a96b-9d137f9a966b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c950754974d748519118d974492f15ce'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:35.655312 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-840baff2-bf1d-4695-a96b-9d137f9a966b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c950754974d748519118d974492f15ce'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:35.673904 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1263/2528] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:09:35 2022] DELETE /identity/v3/users/c950754974d748519118d974492f15ce => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:09:35.850017 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4d856-18d2-4337-b487-5bd5330d32ef service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:35.862312 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4d856-18d2-4337-b487-5bd5330d32ef service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:35.918871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a4463-cf98-4268-a82a-203fa46eb5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:35.919926 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a4463-cf98-4268-a82a-203fa46eb5a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:35.920204 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a4463-cf98-4268-a82a-203fa46eb5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:35.920595 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a4463-cf98-4268-a82a-203fa46eb5a6 admin admin] PATH_INFO: `/v3/projects/0f38af4b388e49958aa17250c181b203` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:35.922031 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a4463-cf98-4268-a82a-203fa46eb5a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f38af4b388e49958aa17250c181b203)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:35.924001 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a4463-cf98-4268-a82a-203fa46eb5a6 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:35.969410 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eb7a4463-cf98-4268-a82a-203fa46eb5a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f38af4b388e49958aa17250c181b203', 'request_id': 'req-eb7a4463-cf98-4268-a82a-203fa46eb5a6'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:35.974629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eb7a4463-cf98-4268-a82a-203fa46eb5a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f38af4b388e49958aa17250c181b203', 'request_id': 'req-eb7a4463-cf98-4268-a82a-203fa46eb5a6'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:35.981060 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eb7a4463-cf98-4268-a82a-203fa46eb5a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f38af4b388e49958aa17250c181b203', 'request_id': 'req-eb7a4463-cf98-4268-a82a-203fa46eb5a6'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:35.993439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eb7a4463-cf98-4268-a82a-203fa46eb5a6 admin admin] The token cache is being invalidate because project 0f38af4b388e49958aa17250c181b203 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:09:35.993729 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eb7a4463-cf98-4268-a82a-203fa46eb5a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:35.994374 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eb7a4463-cf98-4268-a82a-203fa46eb5a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:35.995910 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1266/2529] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:09:35 2022] DELETE /identity/v3/projects/0f38af4b388e49958aa17250c181b203 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:36.001790 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b42fe8-492f-42e0-8170-55b8222647ef None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:36.002218 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b42fe8-492f-42e0-8170-55b8222647ef None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:36.002594 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b42fe8-492f-42e0-8170-55b8222647ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:36.022053 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-01b42fe8-492f-42e0-8170-55b8222647ef None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:36.023248 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-01b42fe8-492f-42e0-8170-55b8222647ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:36.086551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1264/2530] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:36.095609 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f907cb9-6e0b-4610-ab93-7ef4a074079d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:36.095840 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f907cb9-6e0b-4610-ab93-7ef4a074079d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:36.096120 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f907cb9-6e0b-4610-ab93-7ef4a074079d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:36.115566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-6f907cb9-6e0b-4610-ab93-7ef4a074079d None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:36.116789 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6f907cb9-6e0b-4610-ab93-7ef4a074079d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:36.124761 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1267/2531] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:36.130060 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b42fe8-492f-42e0-8170-55b8222647ef None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:36.140538 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b42fe8-492f-42e0-8170-55b8222647ef None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:36.145289 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257320a-0980-42c7-b9c2-abdf871931b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:36.145948 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257320a-0980-42c7-b9c2-abdf871931b5 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:36.146263 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257320a-0980-42c7-b9c2-abdf871931b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:36.146540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257320a-0980-42c7-b9c2-abdf871931b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:36.147260 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f257320a-0980-42c7-b9c2-abdf871931b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:09:36.147587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f257320a-0980-42c7-b9c2-abdf871931b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:36.149209 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f257320a-0980-42c7-b9c2-abdf871931b5 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:36.160760 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1265/2532] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:09:36.165161 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f907cb9-6e0b-4610-ab93-7ef4a074079d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:36.174485 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f907cb9-6e0b-4610-ab93-7ef4a074079d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:36.179680 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e679d-20f1-45d2-a32f-11ac5a62ccd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:36.180324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e679d-20f1-45d2-a32f-11ac5a62ccd5 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:36.180656 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e679d-20f1-45d2-a32f-11ac5a62ccd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:36.180958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e679d-20f1-45d2-a32f-11ac5a62ccd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:36.181650 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e679d-20f1-45d2-a32f-11ac5a62ccd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:36.183605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e679d-20f1-45d2-a32f-11ac5a62ccd5 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:36.214735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1268/2533] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:09:36 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:09:36.219754 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257320a-0980-42c7-b9c2-abdf871931b5 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:36.237280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257320a-0980-42c7-b9c2-abdf871931b5 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:36.297580 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded3240-d937-4f72-a3f1-dab5427c72e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:36.298383 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded3240-d937-4f72-a3f1-dab5427c72e2 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:36.298663 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded3240-d937-4f72-a3f1-dab5427c72e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:36.298943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded3240-d937-4f72-a3f1-dab5427c72e2 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:36.299631 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded3240-d937-4f72-a3f1-dab5427c72e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:36.302162 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded3240-d937-4f72-a3f1-dab5427c72e2 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:36.322069 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1266/2534] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:09:36 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:09:36.326202 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e679d-20f1-45d2-a32f-11ac5a62ccd5 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:36.336255 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e679d-20f1-45d2-a32f-11ac5a62ccd5 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:36.340857 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4b985-42cf-4682-a289-88d2ca3d8c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:36.341511 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4b985-42cf-4682-a289-88d2ca3d8c09 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:36.341795 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4b985-42cf-4682-a289-88d2ca3d8c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:36.342063 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4b985-42cf-4682-a289-88d2ca3d8c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:36.342954 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc4b985-42cf-4682-a289-88d2ca3d8c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:36.344601 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc4b985-42cf-4682-a289-88d2ca3d8c09 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:36.356198 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1269/2535] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:09:36.360079 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded3240-d937-4f72-a3f1-dab5427c72e2 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:36.370377 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded3240-d937-4f72-a3f1-dab5427c72e2 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:36.374898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d34737-0a49-451d-aa0c-95b9a833c5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:36.375561 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d34737-0a49-451d-aa0c-95b9a833c5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:36.375839 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d34737-0a49-451d-aa0c-95b9a833c5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:36.376112 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d34737-0a49-451d-aa0c-95b9a833c5d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:36.376824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d34737-0a49-451d-aa0c-95b9a833c5d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:36.378440 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d34737-0a49-451d-aa0c-95b9a833c5d1 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:36.388474 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1267/2536] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:09:36.392599 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4b985-42cf-4682-a289-88d2ca3d8c09 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:36.402916 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4b985-42cf-4682-a289-88d2ca3d8c09 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:36.407530 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22bf9ef-4460-45f7-8c8c-40d80c18d288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:36.408392 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22bf9ef-4460-45f7-8c8c-40d80c18d288 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:36.408673 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22bf9ef-4460-45f7-8c8c-40d80c18d288 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:36.408944 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22bf9ef-4460-45f7-8c8c-40d80c18d288 admin admin] PATH_INFO: `/v3/projects/61255ec0178640faa3828048625af777/users/5dc01926da9e47f39c4eb0d101831d40/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:36.425253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22bf9ef-4460-45f7-8c8c-40d80c18d288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61255ec0178640faa3828048625af777, user_id=5dc01926da9e47f39c4eb0d101831d40, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:36.428587 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22bf9ef-4460-45f7-8c8c-40d80c18d288 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:36.441724 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1270/2537] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:09:36 2022] PUT /identity/v3/projects/61255ec0178640faa3828048625af777/users/5dc01926da9e47f39c4eb0d101831d40/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:36.451806 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336060d-f3fd-4321-9645-48b5ae4e0d94 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:36.452093 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336060d-f3fd-4321-9645-48b5ae4e0d94 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:36.452370 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336060d-f3fd-4321-9645-48b5ae4e0d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:36.478533 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-1336060d-f3fd-4321-9645-48b5ae4e0d94 None None] MFA Rules not processed for user `5dc01926da9e47f39c4eb0d101831d40`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:36.479711 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1336060d-f3fd-4321-9645-48b5ae4e0d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:36.543478 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1268/2538] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:36.547974 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22bf9ef-4460-45f7-8c8c-40d80c18d288 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:36.611792 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22bf9ef-4460-45f7-8c8c-40d80c18d288 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:36.616395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd56af7-7bb1-45de-bf54-8f6a1b38651a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:36.617059 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd56af7-7bb1-45de-bf54-8f6a1b38651a admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:36.617337 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd56af7-7bb1-45de-bf54-8f6a1b38651a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:36.617609 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd56af7-7bb1-45de-bf54-8f6a1b38651a admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:36.618324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd56af7-7bb1-45de-bf54-8f6a1b38651a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:36.620217 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd56af7-7bb1-45de-bf54-8f6a1b38651a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:36.646054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1271/2539] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:09:36 2022] POST /identity/v3/projects => generated 371 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:09:36.650141 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336060d-f3fd-4321-9645-48b5ae4e0d94 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:36.661261 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336060d-f3fd-4321-9645-48b5ae4e0d94 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:36.716646 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9eb6e-5949-4979-8933-3ad8768e7159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:36.717409 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9eb6e-5949-4979-8933-3ad8768e7159 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:36.717706 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9eb6e-5949-4979-8933-3ad8768e7159 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:36.717985 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9eb6e-5949-4979-8933-3ad8768e7159 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:36.718873 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9eb6e-5949-4979-8933-3ad8768e7159 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:36.720885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9eb6e-5949-4979-8933-3ad8768e7159 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:36.740837 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1269/2540] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:09:36 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:09:36.744479 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd56af7-7bb1-45de-bf54-8f6a1b38651a admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:36.754960 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd56af7-7bb1-45de-bf54-8f6a1b38651a admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:36.761992 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f18ee1-5a4f-43a9-80c9-1f351eb3efe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:36.762773 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f18ee1-5a4f-43a9-80c9-1f351eb3efe7 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:36.763122 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f18ee1-5a4f-43a9-80c9-1f351eb3efe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:36.763467 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f18ee1-5a4f-43a9-80c9-1f351eb3efe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:36.764258 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f18ee1-5a4f-43a9-80c9-1f351eb3efe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:36.766002 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f18ee1-5a4f-43a9-80c9-1f351eb3efe7 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:36.778234 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1272/2541] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:09:36.782338 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9eb6e-5949-4979-8933-3ad8768e7159 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:36.793572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9eb6e-5949-4979-8933-3ad8768e7159 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:36.798641 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a379e36-f4d6-4982-8606-8b4a989292a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:36.799507 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a379e36-f4d6-4982-8606-8b4a989292a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:36.799785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a379e36-f4d6-4982-8606-8b4a989292a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:36.800056 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a379e36-f4d6-4982-8606-8b4a989292a7 admin admin] PATH_INFO: `/v3/projects/663748103e0d422caca27dbd8befcc99/users/c5be690dbffc4b57b8210bb4a41e97c8/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:36.814275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a379e36-f4d6-4982-8606-8b4a989292a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=663748103e0d422caca27dbd8befcc99, user_id=c5be690dbffc4b57b8210bb4a41e97c8, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:36.816919 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a379e36-f4d6-4982-8606-8b4a989292a7 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:36.830291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1270/2542] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:09:36 2022] PUT /identity/v3/projects/663748103e0d422caca27dbd8befcc99/users/c5be690dbffc4b57b8210bb4a41e97c8/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:36.834952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f18ee1-5a4f-43a9-80c9-1f351eb3efe7 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:36.845295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f18ee1-5a4f-43a9-80c9-1f351eb3efe7 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:36.906276 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6024a7-d30e-4cf7-b54d-773be1d11a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:36.906929 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6024a7-d30e-4cf7-b54d-773be1d11a3d admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:36.907190 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6024a7-d30e-4cf7-b54d-773be1d11a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:36.907465 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6024a7-d30e-4cf7-b54d-773be1d11a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:36.908200 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6024a7-d30e-4cf7-b54d-773be1d11a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:36.909782 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6024a7-d30e-4cf7-b54d-773be1d11a3d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:36.919997 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1273/2543] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:09:36.924210 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a379e36-f4d6-4982-8606-8b4a989292a7 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:36.935176 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a379e36-f4d6-4982-8606-8b4a989292a7 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:36.939982 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2edb8-7ce9-4d67-8266-e9cc917c67b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:36.940937 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2edb8-7ce9-4d67-8266-e9cc917c67b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:36.941215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2edb8-7ce9-4d67-8266-e9cc917c67b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:36.941506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2edb8-7ce9-4d67-8266-e9cc917c67b3 admin admin] PATH_INFO: `/v3/projects/663748103e0d422caca27dbd8befcc99/users/c5be690dbffc4b57b8210bb4a41e97c8/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:36.943538 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2edb8-7ce9-4d67-8266-e9cc917c67b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=663748103e0d422caca27dbd8befcc99, user_id=c5be690dbffc4b57b8210bb4a41e97c8, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:36.946787 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2edb8-7ce9-4d67-8266-e9cc917c67b3 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:36.957264 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1271/2544] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:09:36 2022] PUT /identity/v3/projects/663748103e0d422caca27dbd8befcc99/users/c5be690dbffc4b57b8210bb4a41e97c8/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:36.969280 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a76a0-7b75-4b81-a181-2aaa913faaf7 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:36.969568 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a76a0-7b75-4b81-a181-2aaa913faaf7 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:36.969844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a76a0-7b75-4b81-a181-2aaa913faaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:36.995629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5b6a76a0-7b75-4b81-a181-2aaa913faaf7 None None] MFA Rules not processed for user `c5be690dbffc4b57b8210bb4a41e97c8`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:36.997097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5b6a76a0-7b75-4b81-a181-2aaa913faaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:37.064836 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1274/2545] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:36 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:37.077521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2edb8-7ce9-4d67-8266-e9cc917c67b3 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:37.079573 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-cec2edb8-7ce9-4d67-8266-e9cc917c67b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:37.137164 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2edb8-7ce9-4d67-8266-e9cc917c67b3 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:37.141348 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2089d-f675-44f5-8793-aea284d13d4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:37.142004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2089d-f675-44f5-8793-aea284d13d4d service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:37.142311 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2089d-f675-44f5-8793-aea284d13d4d service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:37.142584 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2089d-f675-44f5-8793-aea284d13d4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:37.152625 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2089d-f675-44f5-8793-aea284d13d4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:37.154044 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2089d-f675-44f5-8793-aea284d13d4d service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:37.164896 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1272/2546] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:09:37 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:09:37.249266 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a76a0-7b75-4b81-a181-2aaa913faaf7 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:37.259441 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a76a0-7b75-4b81-a181-2aaa913faaf7 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:37.263119 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b69bec-e9f0-494c-ad05-01feaab8724a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:37.263732 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b69bec-e9f0-494c-ad05-01feaab8724a service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:37.264007 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b69bec-e9f0-494c-ad05-01feaab8724a service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:37.264277 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b69bec-e9f0-494c-ad05-01feaab8724a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:37.319729 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b69bec-e9f0-494c-ad05-01feaab8724a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:37.321162 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b69bec-e9f0-494c-ad05-01feaab8724a service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:37.330194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1275/2547] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:09:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:09:49.658764 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2089d-f675-44f5-8793-aea284d13d4d service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:49.660163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1ee2089d-f675-44f5-8793-aea284d13d4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:49.721986 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2089d-f675-44f5-8793-aea284d13d4d service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:49.728472 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5291f-a192-4256-b2fa-dde8b079cfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:49.729316 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5291f-a192-4256-b2fa-dde8b079cfaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:49.729589 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5291f-a192-4256-b2fa-dde8b079cfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:49.729855 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5291f-a192-4256-b2fa-dde8b079cfaf admin admin] PATH_INFO: `/v3/users/ef5b51a6fdaa4a6286e1e9f007dac776` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:49.731301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5291f-a192-4256-b2fa-dde8b079cfaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef5b51a6fdaa4a6286e1e9f007dac776)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:49.733197 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5291f-a192-4256-b2fa-dde8b079cfaf admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:49.776710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d4d5291f-a192-4256-b2fa-dde8b079cfaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef5b51a6fdaa4a6286e1e9f007dac776'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:49.783719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d4d5291f-a192-4256-b2fa-dde8b079cfaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef5b51a6fdaa4a6286e1e9f007dac776'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:49.784357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d4d5291f-a192-4256-b2fa-dde8b079cfaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef5b51a6fdaa4a6286e1e9f007dac776'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:49.790528 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d4d5291f-a192-4256-b2fa-dde8b079cfaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef5b51a6fdaa4a6286e1e9f007dac776'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:49.791408 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d4d5291f-a192-4256-b2fa-dde8b079cfaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef5b51a6fdaa4a6286e1e9f007dac776'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:49.807308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1273/2548] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:09:49 2022] DELETE /identity/v3/users/ef5b51a6fdaa4a6286e1e9f007dac776 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:49.811837 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099d258-c3a9-4aff-adca-e37ba4437b9a None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:49.812315 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099d258-c3a9-4aff-adca-e37ba4437b9a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:49.812429 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099d258-c3a9-4aff-adca-e37ba4437b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:49.829118 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5099d258-c3a9-4aff-adca-e37ba4437b9a None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:49.830319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5099d258-c3a9-4aff-adca-e37ba4437b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:49.894517 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1276/2549] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:49.905371 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5291f-a192-4256-b2fa-dde8b079cfaf admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:49.906840 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d4d5291f-a192-4256-b2fa-dde8b079cfaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:49.967100 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5291f-a192-4256-b2fa-dde8b079cfaf admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:49.970786 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26661483-c0f3-4ecf-ae17-3bf6c70f2351 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:49.971422 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26661483-c0f3-4ecf-ae17-3bf6c70f2351 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:49.971700 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26661483-c0f3-4ecf-ae17-3bf6c70f2351 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:49.972026 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26661483-c0f3-4ecf-ae17-3bf6c70f2351 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:49.981847 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26661483-c0f3-4ecf-ae17-3bf6c70f2351 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:49.983370 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26661483-c0f3-4ecf-ae17-3bf6c70f2351 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:49.993535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1274/2550] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:09:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:09:50.267185 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099d258-c3a9-4aff-adca-e37ba4437b9a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:50.268606 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5099d258-c3a9-4aff-adca-e37ba4437b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:50.280647 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099d258-c3a9-4aff-adca-e37ba4437b9a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:50.285292 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c9a62-ecf0-457e-8ae9-9c99ea92d754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:50.286172 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c9a62-ecf0-457e-8ae9-9c99ea92d754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:50.286460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c9a62-ecf0-457e-8ae9-9c99ea92d754 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:50.286739 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c9a62-ecf0-457e-8ae9-9c99ea92d754 admin admin] PATH_INFO: `/v3/projects/ea4b83efa8ad41879ef0d18492aaecbf` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:50.287937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c9a62-ecf0-457e-8ae9-9c99ea92d754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea4b83efa8ad41879ef0d18492aaecbf)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:50.289876 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c9a62-ecf0-457e-8ae9-9c99ea92d754 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:50.335037 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-149c9a62-ecf0-457e-8ae9-9c99ea92d754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea4b83efa8ad41879ef0d18492aaecbf', 'request_id': 'req-149c9a62-ecf0-457e-8ae9-9c99ea92d754'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:50.340490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-149c9a62-ecf0-457e-8ae9-9c99ea92d754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea4b83efa8ad41879ef0d18492aaecbf', 'request_id': 'req-149c9a62-ecf0-457e-8ae9-9c99ea92d754'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:50.346601 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-149c9a62-ecf0-457e-8ae9-9c99ea92d754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea4b83efa8ad41879ef0d18492aaecbf', 'request_id': 'req-149c9a62-ecf0-457e-8ae9-9c99ea92d754'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:50.360019 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-149c9a62-ecf0-457e-8ae9-9c99ea92d754 admin admin] The token cache is being invalidate because project ea4b83efa8ad41879ef0d18492aaecbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:09:50.360313 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-149c9a62-ecf0-457e-8ae9-9c99ea92d754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:50.360928 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-149c9a62-ecf0-457e-8ae9-9c99ea92d754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:50.362520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1277/2551] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:09:50 2022] DELETE /identity/v3/projects/ea4b83efa8ad41879ef0d18492aaecbf => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:50.368033 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733ee6e-a813-435c-8743-3e59c8866aaf None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:50.368316 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733ee6e-a813-435c-8743-3e59c8866aaf None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:50.368589 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733ee6e-a813-435c-8743-3e59c8866aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:50.385356 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-1733ee6e-a813-435c-8743-3e59c8866aaf None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:50.386549 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1733ee6e-a813-435c-8743-3e59c8866aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:50.441475 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1275/2552] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:50.450710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4681b8-262b-4bc3-8e97-56bb538a348c None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:50.450992 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4681b8-262b-4bc3-8e97-56bb538a348c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:50.451276 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4681b8-262b-4bc3-8e97-56bb538a348c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:50.468885 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-7c4681b8-262b-4bc3-8e97-56bb538a348c None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:50.470074 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7c4681b8-262b-4bc3-8e97-56bb538a348c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:50.478896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1278/2553] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:50.483183 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733ee6e-a813-435c-8743-3e59c8866aaf None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:50.493023 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733ee6e-a813-435c-8743-3e59c8866aaf None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:50.497482 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf837a6-2c8f-44d8-a40a-5c47bdaea1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:50.498140 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf837a6-2c8f-44d8-a40a-5c47bdaea1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:50.498421 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf837a6-2c8f-44d8-a40a-5c47bdaea1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:50.498690 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf837a6-2c8f-44d8-a40a-5c47bdaea1d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:50.499419 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf837a6-2c8f-44d8-a40a-5c47bdaea1d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:09:50.499717 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf837a6-2c8f-44d8-a40a-5c47bdaea1d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:50.501287 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf837a6-2c8f-44d8-a40a-5c47bdaea1d5 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:50.512153 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1276/2554] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:09:50.516392 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4681b8-262b-4bc3-8e97-56bb538a348c None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:50.525337 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4681b8-262b-4bc3-8e97-56bb538a348c None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:50.529883 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626adbb1-2a92-4eff-8084-bbd5a3b3ca1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:50.530557 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626adbb1-2a92-4eff-8084-bbd5a3b3ca1e admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:50.530838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626adbb1-2a92-4eff-8084-bbd5a3b3ca1e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:50.531110 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626adbb1-2a92-4eff-8084-bbd5a3b3ca1e admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:50.531782 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626adbb1-2a92-4eff-8084-bbd5a3b3ca1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:50.533655 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626adbb1-2a92-4eff-8084-bbd5a3b3ca1e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:50.558735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1279/2555] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:09:50 2022] POST /identity/v3/projects => generated 377 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:09:50.562812 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf837a6-2c8f-44d8-a40a-5c47bdaea1d5 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:50.572499 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf837a6-2c8f-44d8-a40a-5c47bdaea1d5 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:50.628155 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f7300-636f-49ee-a6a7-5853784e5578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:50.628725 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f7300-636f-49ee-a6a7-5853784e5578 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:50.629004 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f7300-636f-49ee-a6a7-5853784e5578 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:50.629280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f7300-636f-49ee-a6a7-5853784e5578 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:50.629966 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f7300-636f-49ee-a6a7-5853784e5578 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:50.631917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f7300-636f-49ee-a6a7-5853784e5578 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:50.651361 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1277/2556] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:09:50 2022] POST /identity/v3/users => generated 289 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:09:50.655414 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626adbb1-2a92-4eff-8084-bbd5a3b3ca1e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:50.665878 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626adbb1-2a92-4eff-8084-bbd5a3b3ca1e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:50.670433 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bdef4b-3389-4199-9a38-4cf615d37269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:50.671077 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdef4b-3389-4199-9a38-4cf615d37269 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:50.671357 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdef4b-3389-4199-9a38-4cf615d37269 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:50.671629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdef4b-3389-4199-9a38-4cf615d37269 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:50.672342 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bdef4b-3389-4199-9a38-4cf615d37269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:50.673917 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bdef4b-3389-4199-9a38-4cf615d37269 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:50.684514 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1280/2557] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:09:50.688168 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f7300-636f-49ee-a6a7-5853784e5578 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:50.697803 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f7300-636f-49ee-a6a7-5853784e5578 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:50.702314 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9ddc5-d78c-4939-b5bf-5b3ab0a59754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:50.702955 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9ddc5-d78c-4939-b5bf-5b3ab0a59754 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:50.703236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9ddc5-d78c-4939-b5bf-5b3ab0a59754 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:50.703509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9ddc5-d78c-4939-b5bf-5b3ab0a59754 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:50.704225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9ddc5-d78c-4939-b5bf-5b3ab0a59754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:50.705792 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9ddc5-d78c-4939-b5bf-5b3ab0a59754 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:50.715733 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1278/2558] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:09:50.719895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bdef4b-3389-4199-9a38-4cf615d37269 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:50.730042 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bdef4b-3389-4199-9a38-4cf615d37269 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:50.734854 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d786aff-4a5d-46c6-9a50-8f433069ebb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:50.735710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d786aff-4a5d-46c6-9a50-8f433069ebb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:50.735987 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d786aff-4a5d-46c6-9a50-8f433069ebb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:50.736260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d786aff-4a5d-46c6-9a50-8f433069ebb5 admin admin] PATH_INFO: `/v3/projects/4c6905a729184fffa958f2a5f9bd3a07/users/119ab9fc7b254e5a9e68f70d90c017f3/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:50.750065 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d786aff-4a5d-46c6-9a50-8f433069ebb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c6905a729184fffa958f2a5f9bd3a07, user_id=119ab9fc7b254e5a9e68f70d90c017f3, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:50.752704 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d786aff-4a5d-46c6-9a50-8f433069ebb5 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:50.765116 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1281/2559] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:09:50 2022] PUT /identity/v3/projects/4c6905a729184fffa958f2a5f9bd3a07/users/119ab9fc7b254e5a9e68f70d90c017f3/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:50.775667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff5c8c-a7d5-4bbf-857a-8d8af669b5f1 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:50.775958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff5c8c-a7d5-4bbf-857a-8d8af669b5f1 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:50.776253 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff5c8c-a7d5-4bbf-857a-8d8af669b5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:50.807240 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-38ff5c8c-a7d5-4bbf-857a-8d8af669b5f1 None None] MFA Rules not processed for user `119ab9fc7b254e5a9e68f70d90c017f3`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:50.808434 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-38ff5c8c-a7d5-4bbf-857a-8d8af669b5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:50.870339 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1279/2560] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:50.874807 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d786aff-4a5d-46c6-9a50-8f433069ebb5 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:50.933909 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d786aff-4a5d-46c6-9a50-8f433069ebb5 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:50.938499 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b6508-e9ac-4ea2-ae5e-eebd4f22bd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:50.939138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b6508-e9ac-4ea2-ae5e-eebd4f22bd29 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:50.939416 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b6508-e9ac-4ea2-ae5e-eebd4f22bd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:50.939688 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b6508-e9ac-4ea2-ae5e-eebd4f22bd29 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:50.940375 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b6508-e9ac-4ea2-ae5e-eebd4f22bd29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:50.942298 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b6508-e9ac-4ea2-ae5e-eebd4f22bd29 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:50.966595 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1282/2561] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:09:50 2022] POST /identity/v3/projects => generated 375 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:09:50.970489 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff5c8c-a7d5-4bbf-857a-8d8af669b5f1 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:50.981682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff5c8c-a7d5-4bbf-857a-8d8af669b5f1 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:51.033384 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7d9f7-d840-4fc0-8c3f-fb37af01584c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:51.034032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7d9f7-d840-4fc0-8c3f-fb37af01584c admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:51.034342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7d9f7-d840-4fc0-8c3f-fb37af01584c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:51.034619 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7d9f7-d840-4fc0-8c3f-fb37af01584c admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:51.035298 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7d9f7-d840-4fc0-8c3f-fb37af01584c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:51.037219 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7d9f7-d840-4fc0-8c3f-fb37af01584c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:51.056321 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1280/2562] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:09:50 2022] POST /identity/v3/users => generated 288 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:09:51.060427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b6508-e9ac-4ea2-ae5e-eebd4f22bd29 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:51.070357 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b6508-e9ac-4ea2-ae5e-eebd4f22bd29 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:51.074878 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2364c19-e373-479f-ba49-9d672302ae8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:51.075524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2364c19-e373-479f-ba49-9d672302ae8e admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:51.075803 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2364c19-e373-479f-ba49-9d672302ae8e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:51.076079 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2364c19-e373-479f-ba49-9d672302ae8e admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:51.076790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2364c19-e373-479f-ba49-9d672302ae8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:51.078403 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2364c19-e373-479f-ba49-9d672302ae8e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:51.088969 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1283/2563] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:09:51.092944 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7d9f7-d840-4fc0-8c3f-fb37af01584c admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:51.103278 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7d9f7-d840-4fc0-8c3f-fb37af01584c admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:51.108030 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d557a5-40a5-456f-b16b-e8cf7b0c69f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:51.108667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d557a5-40a5-456f-b16b-e8cf7b0c69f1 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:51.108939 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d557a5-40a5-456f-b16b-e8cf7b0c69f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:51.109207 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d557a5-40a5-456f-b16b-e8cf7b0c69f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:51.109911 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d557a5-40a5-456f-b16b-e8cf7b0c69f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:51.111499 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d557a5-40a5-456f-b16b-e8cf7b0c69f1 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:51.121139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1281/2564] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:09:51.125133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2364c19-e373-479f-ba49-9d672302ae8e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:51.135329 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2364c19-e373-479f-ba49-9d672302ae8e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:51.139970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1e499-fa7a-4577-879e-4614503df82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:51.140829 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1e499-fa7a-4577-879e-4614503df82c admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:51.141098 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1e499-fa7a-4577-879e-4614503df82c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:51.141368 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1e499-fa7a-4577-879e-4614503df82c admin admin] PATH_INFO: `/v3/projects/49535bb851994866901e2b4ff0ed6838/users/95314ba19697456986e91ebc0fc13a28/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:51.157407 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1e499-fa7a-4577-879e-4614503df82c admin admin] RBAC: Authorizing `identity:create_grant(project_id=49535bb851994866901e2b4ff0ed6838, user_id=95314ba19697456986e91ebc0fc13a28, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:51.160153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1e499-fa7a-4577-879e-4614503df82c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:51.173165 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1284/2565] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:09:51 2022] PUT /identity/v3/projects/49535bb851994866901e2b4ff0ed6838/users/95314ba19697456986e91ebc0fc13a28/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:09:51.183402 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba4b5c-c341-4819-a2b7-18b597a091b5 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:51.183690 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba4b5c-c341-4819-a2b7-18b597a091b5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:51.184002 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba4b5c-c341-4819-a2b7-18b597a091b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:51.210191 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-cbba4b5c-c341-4819-a2b7-18b597a091b5 None None] MFA Rules not processed for user `95314ba19697456986e91ebc0fc13a28`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:51.212002 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-cbba4b5c-c341-4819-a2b7-18b597a091b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:51.274998 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1282/2566] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:51.279703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1e499-fa7a-4577-879e-4614503df82c admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:51.344648 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1e499-fa7a-4577-879e-4614503df82c admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:51.349554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e3890-b528-4d06-ae23-962f17c9290c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:51.350297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e3890-b528-4d06-ae23-962f17c9290c admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:51.350582 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e3890-b528-4d06-ae23-962f17c9290c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:51.350857 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e3890-b528-4d06-ae23-962f17c9290c admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:51.351622 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e3890-b528-4d06-ae23-962f17c9290c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:51.353574 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e3890-b528-4d06-ae23-962f17c9290c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:51.381050 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1285/2567] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:09:51 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:09:51.385880 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba4b5c-c341-4819-a2b7-18b597a091b5 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:51.396925 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba4b5c-c341-4819-a2b7-18b597a091b5 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:51.455554 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880eab2-a1ab-4ebd-95d6-38de627f9100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:51.456416 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880eab2-a1ab-4ebd-95d6-38de627f9100 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:51.456723 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880eab2-a1ab-4ebd-95d6-38de627f9100 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:51.457016 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880eab2-a1ab-4ebd-95d6-38de627f9100 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:51.457708 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880eab2-a1ab-4ebd-95d6-38de627f9100 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:51.459720 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880eab2-a1ab-4ebd-95d6-38de627f9100 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:51.483871 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1283/2568] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:09:51 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:09:51.484955 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e3890-b528-4d06-ae23-962f17c9290c admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:51.496134 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e3890-b528-4d06-ae23-962f17c9290c admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:51.502165 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda24263-a619-4f04-ac16-23856a31c93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:51.503051 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda24263-a619-4f04-ac16-23856a31c93c admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:51.503330 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda24263-a619-4f04-ac16-23856a31c93c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:51.503604 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda24263-a619-4f04-ac16-23856a31c93c admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:51.504472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda24263-a619-4f04-ac16-23856a31c93c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:51.506422 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda24263-a619-4f04-ac16-23856a31c93c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:51.517768 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1286/2569] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:09:51.522275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880eab2-a1ab-4ebd-95d6-38de627f9100 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:51.532875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880eab2-a1ab-4ebd-95d6-38de627f9100 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:51.537552 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9df5e-43a9-4e38-b2c1-310cdf71fe9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:51.538491 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9df5e-43a9-4e38-b2c1-310cdf71fe9f admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:51.538786 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9df5e-43a9-4e38-b2c1-310cdf71fe9f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:51.539174 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9df5e-43a9-4e38-b2c1-310cdf71fe9f admin admin] PATH_INFO: `/v3/projects/0c8d1b988b6341169af9b3c19fe2a171/users/eadcee324bec4d0b8238e4e33c331636/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:51.556587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9df5e-43a9-4e38-b2c1-310cdf71fe9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c8d1b988b6341169af9b3c19fe2a171, user_id=eadcee324bec4d0b8238e4e33c331636, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:51.560052 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9df5e-43a9-4e38-b2c1-310cdf71fe9f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:51.582244 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1284/2570] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:09:51 2022] PUT /identity/v3/projects/0c8d1b988b6341169af9b3c19fe2a171/users/eadcee324bec4d0b8238e4e33c331636/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:09:51.586523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda24263-a619-4f04-ac16-23856a31c93c admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:51.596580 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda24263-a619-4f04-ac16-23856a31c93c admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:51.671022 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4b3a3-acdb-4d43-8037-382217a29f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:51.671683 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4b3a3-acdb-4d43-8037-382217a29f17 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:51.671964 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4b3a3-acdb-4d43-8037-382217a29f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:51.672236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4b3a3-acdb-4d43-8037-382217a29f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:51.672950 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f4b3a3-acdb-4d43-8037-382217a29f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:51.674555 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f4b3a3-acdb-4d43-8037-382217a29f17 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:51.684959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1287/2571] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:09:51.689188 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9df5e-43a9-4e38-b2c1-310cdf71fe9f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:51.700069 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9df5e-43a9-4e38-b2c1-310cdf71fe9f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:51.705250 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86a9a4-17e1-43cf-ba55-b7cc46f651ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:51.706141 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86a9a4-17e1-43cf-ba55-b7cc46f651ae admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:51.706458 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86a9a4-17e1-43cf-ba55-b7cc46f651ae admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:51.706731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86a9a4-17e1-43cf-ba55-b7cc46f651ae admin admin] PATH_INFO: `/v3/projects/0c8d1b988b6341169af9b3c19fe2a171/users/eadcee324bec4d0b8238e4e33c331636/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:51.708674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd86a9a4-17e1-43cf-ba55-b7cc46f651ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c8d1b988b6341169af9b3c19fe2a171, user_id=eadcee324bec4d0b8238e4e33c331636, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:51.711446 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd86a9a4-17e1-43cf-ba55-b7cc46f651ae admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:51.722221 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1285/2572] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:09:51 2022] PUT /identity/v3/projects/0c8d1b988b6341169af9b3c19fe2a171/users/eadcee324bec4d0b8238e4e33c331636/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:51.733870 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd25492-36b2-45da-a956-7138f3763002 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:51.734094 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd25492-36b2-45da-a956-7138f3763002 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:51.734405 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd25492-36b2-45da-a956-7138f3763002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:51.760881 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-6dd25492-36b2-45da-a956-7138f3763002 None None] MFA Rules not processed for user `eadcee324bec4d0b8238e4e33c331636`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:51.762071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6dd25492-36b2-45da-a956-7138f3763002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:51.829005 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1288/2573] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:51 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:51.840746 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86a9a4-17e1-43cf-ba55-b7cc46f651ae admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:51.843085 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-dd86a9a4-17e1-43cf-ba55-b7cc46f651ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:51.899149 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86a9a4-17e1-43cf-ba55-b7cc46f651ae admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:51.902789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111309a2-d2f7-41ae-a66e-6feba4bf297a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:51.903417 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111309a2-d2f7-41ae-a66e-6feba4bf297a service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:51.903695 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111309a2-d2f7-41ae-a66e-6feba4bf297a service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:51.903968 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111309a2-d2f7-41ae-a66e-6feba4bf297a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:51.958234 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111309a2-d2f7-41ae-a66e-6feba4bf297a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:51.959659 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111309a2-d2f7-41ae-a66e-6feba4bf297a service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:51.973594 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1286/2574] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:09:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:09:53.440320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd25492-36b2-45da-a956-7138f3763002 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:53.449767 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd25492-36b2-45da-a956-7138f3763002 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:53.453324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab401f-fc05-40de-a8f3-6bd340467f34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:53.453953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab401f-fc05-40de-a8f3-6bd340467f34 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:53.454265 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab401f-fc05-40de-a8f3-6bd340467f34 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:53.454537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab401f-fc05-40de-a8f3-6bd340467f34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:53.509285 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aab401f-fc05-40de-a8f3-6bd340467f34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:53.510679 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aab401f-fc05-40de-a8f3-6bd340467f34 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:53.519270 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1289/2575] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:09:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:09:54.033904 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111309a2-d2f7-41ae-a66e-6feba4bf297a service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:54.035337 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-111309a2-d2f7-41ae-a66e-6feba4bf297a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:54.046958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111309a2-d2f7-41ae-a66e-6feba4bf297a service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:54.050914 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7949a-eb16-405a-88f1-538e1ba633b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:54.051584 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7949a-eb16-405a-88f1-538e1ba633b0 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:54.051865 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7949a-eb16-405a-88f1-538e1ba633b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:54.052144 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7949a-eb16-405a-88f1-538e1ba633b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:54.061351 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7949a-eb16-405a-88f1-538e1ba633b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:54.063045 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7949a-eb16-405a-88f1-538e1ba633b0 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:54.072401 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1287/2576] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:09:54 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:09:57.892397 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab401f-fc05-40de-a8f3-6bd340467f34 service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:57.953370 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab401f-fc05-40de-a8f3-6bd340467f34 service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:57.957861 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa05c77-a99a-4442-9c91-5376ad7fc0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:57.958736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa05c77-a99a-4442-9c91-5376ad7fc0db admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:57.959014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa05c77-a99a-4442-9c91-5376ad7fc0db admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:57.959283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa05c77-a99a-4442-9c91-5376ad7fc0db admin admin] PATH_INFO: `/v3/users/119ab9fc7b254e5a9e68f70d90c017f3` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:57.960443 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa05c77-a99a-4442-9c91-5376ad7fc0db admin admin] RBAC: Authorizing `identity:delete_user(user_id=119ab9fc7b254e5a9e68f70d90c017f3)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:57.962374 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa05c77-a99a-4442-9c91-5376ad7fc0db admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:58.006486 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eaa05c77-a99a-4442-9c91-5376ad7fc0db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '119ab9fc7b254e5a9e68f70d90c017f3'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:58.013000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eaa05c77-a99a-4442-9c91-5376ad7fc0db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '119ab9fc7b254e5a9e68f70d90c017f3'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:58.020034 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eaa05c77-a99a-4442-9c91-5376ad7fc0db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '119ab9fc7b254e5a9e68f70d90c017f3'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:58.020705 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eaa05c77-a99a-4442-9c91-5376ad7fc0db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '119ab9fc7b254e5a9e68f70d90c017f3'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:58.032312 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eaa05c77-a99a-4442-9c91-5376ad7fc0db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '119ab9fc7b254e5a9e68f70d90c017f3'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:58.038034 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1290/2577] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:09:57 2022] DELETE /identity/v3/users/119ab9fc7b254e5a9e68f70d90c017f3 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:58.042987 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78368e20-3891-42a3-a41b-5ca61750f97b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:58.043503 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78368e20-3891-42a3-a41b-5ca61750f97b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:58.043603 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78368e20-3891-42a3-a41b-5ca61750f97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:58.063389 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-78368e20-3891-42a3-a41b-5ca61750f97b None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:58.064565 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-78368e20-3891-42a3-a41b-5ca61750f97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:58.125108 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1288/2578] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:58.134856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa05c77-a99a-4442-9c91-5376ad7fc0db admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:58.136811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-eaa05c77-a99a-4442-9c91-5376ad7fc0db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:58.192662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa05c77-a99a-4442-9c91-5376ad7fc0db admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:58.196306 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ce381-d771-437f-9981-d8da3d44728b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:58.196939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ce381-d771-437f-9981-d8da3d44728b service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:58.197220 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ce381-d771-437f-9981-d8da3d44728b service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:58.197493 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ce381-d771-437f-9981-d8da3d44728b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:58.206220 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ce381-d771-437f-9981-d8da3d44728b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:58.207633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ce381-d771-437f-9981-d8da3d44728b service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:58.217300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1291/2579] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:09:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:09:58.478588 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78368e20-3891-42a3-a41b-5ca61750f97b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:58.480034 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-78368e20-3891-42a3-a41b-5ca61750f97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:58.498688 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78368e20-3891-42a3-a41b-5ca61750f97b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:58.503409 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89bd45-47aa-4d5b-bd52-5fa8e1d308f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:58.504234 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89bd45-47aa-4d5b-bd52-5fa8e1d308f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:58.504529 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89bd45-47aa-4d5b-bd52-5fa8e1d308f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:58.504885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89bd45-47aa-4d5b-bd52-5fa8e1d308f8 admin admin] PATH_INFO: `/v3/projects/4c6905a729184fffa958f2a5f9bd3a07` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:58.506011 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab89bd45-47aa-4d5b-bd52-5fa8e1d308f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c6905a729184fffa958f2a5f9bd3a07)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:58.508381 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab89bd45-47aa-4d5b-bd52-5fa8e1d308f8 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:58.565336 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ab89bd45-47aa-4d5b-bd52-5fa8e1d308f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c6905a729184fffa958f2a5f9bd3a07', 'request_id': 'req-ab89bd45-47aa-4d5b-bd52-5fa8e1d308f8'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:58.574088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ab89bd45-47aa-4d5b-bd52-5fa8e1d308f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c6905a729184fffa958f2a5f9bd3a07', 'request_id': 'req-ab89bd45-47aa-4d5b-bd52-5fa8e1d308f8'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:58.587757 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ab89bd45-47aa-4d5b-bd52-5fa8e1d308f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c6905a729184fffa958f2a5f9bd3a07', 'request_id': 'req-ab89bd45-47aa-4d5b-bd52-5fa8e1d308f8'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:58.605194 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ab89bd45-47aa-4d5b-bd52-5fa8e1d308f8 admin admin] The token cache is being invalidate because project 4c6905a729184fffa958f2a5f9bd3a07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:09:58.605591 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ab89bd45-47aa-4d5b-bd52-5fa8e1d308f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:58.606558 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ab89bd45-47aa-4d5b-bd52-5fa8e1d308f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:58.608794 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1289/2580] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:09:58 2022] DELETE /identity/v3/projects/4c6905a729184fffa958f2a5f9bd3a07 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:58.613562 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ce381-d771-437f-9981-d8da3d44728b service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:58.630601 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ce381-d771-437f-9981-d8da3d44728b service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:58.700640 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e1b4ae-28b3-455a-ab5f-58bd271294f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:58.701863 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1b4ae-28b3-455a-ab5f-58bd271294f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:58.702175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1b4ae-28b3-455a-ab5f-58bd271294f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:58.702456 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1b4ae-28b3-455a-ab5f-58bd271294f2 admin admin] PATH_INFO: `/v3/users/95314ba19697456986e91ebc0fc13a28` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:58.703650 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e1b4ae-28b3-455a-ab5f-58bd271294f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95314ba19697456986e91ebc0fc13a28)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:58.705640 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e1b4ae-28b3-455a-ab5f-58bd271294f2 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:58.757729 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-23e1b4ae-28b3-455a-ab5f-58bd271294f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95314ba19697456986e91ebc0fc13a28'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:58.765485 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-23e1b4ae-28b3-455a-ab5f-58bd271294f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95314ba19697456986e91ebc0fc13a28'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:58.772908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-23e1b4ae-28b3-455a-ab5f-58bd271294f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95314ba19697456986e91ebc0fc13a28'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:58.773713 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-23e1b4ae-28b3-455a-ab5f-58bd271294f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95314ba19697456986e91ebc0fc13a28'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:58.787092 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-23e1b4ae-28b3-455a-ab5f-58bd271294f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95314ba19697456986e91ebc0fc13a28'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:58.793233 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1292/2581] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:09:58 2022] DELETE /identity/v3/users/95314ba19697456986e91ebc0fc13a28 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:59.054830 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89bd45-47aa-4d5b-bd52-5fa8e1d308f8 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:59.056951 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ab89bd45-47aa-4d5b-bd52-5fa8e1d308f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:59.070153 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89bd45-47aa-4d5b-bd52-5fa8e1d308f8 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:59.128755 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e66e84-e4f5-4999-8894-c0d1ca146dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:59.129611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e66e84-e4f5-4999-8894-c0d1ca146dd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:59.129919 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e66e84-e4f5-4999-8894-c0d1ca146dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:59.130216 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e66e84-e4f5-4999-8894-c0d1ca146dd4 admin admin] PATH_INFO: `/v3/projects/49535bb851994866901e2b4ff0ed6838` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:59.131399 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e66e84-e4f5-4999-8894-c0d1ca146dd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49535bb851994866901e2b4ff0ed6838)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:59.133290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e66e84-e4f5-4999-8894-c0d1ca146dd4 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:59.181292 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-28e66e84-e4f5-4999-8894-c0d1ca146dd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49535bb851994866901e2b4ff0ed6838', 'request_id': 'req-28e66e84-e4f5-4999-8894-c0d1ca146dd4'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:59.187342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-28e66e84-e4f5-4999-8894-c0d1ca146dd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49535bb851994866901e2b4ff0ed6838', 'request_id': 'req-28e66e84-e4f5-4999-8894-c0d1ca146dd4'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:59.192632 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-28e66e84-e4f5-4999-8894-c0d1ca146dd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49535bb851994866901e2b4ff0ed6838', 'request_id': 'req-28e66e84-e4f5-4999-8894-c0d1ca146dd4'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:59.205516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-28e66e84-e4f5-4999-8894-c0d1ca146dd4 admin admin] The token cache is being invalidate because project 49535bb851994866901e2b4ff0ed6838 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:09:59.205805 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-28e66e84-e4f5-4999-8894-c0d1ca146dd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:59.206439 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-28e66e84-e4f5-4999-8894-c0d1ca146dd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:59.207976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1290/2582] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:09:59 2022] DELETE /identity/v3/projects/49535bb851994866901e2b4ff0ed6838 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:59.212214 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e1b4ae-28b3-455a-ab5f-58bd271294f2 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:59.213720 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-23e1b4ae-28b3-455a-ab5f-58bd271294f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:59.226012 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e1b4ae-28b3-455a-ab5f-58bd271294f2 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:59.280744 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67393c20-9236-4207-a90d-60734021b561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:59.281601 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67393c20-9236-4207-a90d-60734021b561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:59.281882 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67393c20-9236-4207-a90d-60734021b561 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:59.282184 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67393c20-9236-4207-a90d-60734021b561 admin admin] PATH_INFO: `/v3/users/eadcee324bec4d0b8238e4e33c331636` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:59.283350 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67393c20-9236-4207-a90d-60734021b561 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eadcee324bec4d0b8238e4e33c331636)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:59.285255 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67393c20-9236-4207-a90d-60734021b561 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:59.331263 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-67393c20-9236-4207-a90d-60734021b561 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eadcee324bec4d0b8238e4e33c331636'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:59.337649 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-67393c20-9236-4207-a90d-60734021b561 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eadcee324bec4d0b8238e4e33c331636'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:59.344823 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-67393c20-9236-4207-a90d-60734021b561 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eadcee324bec4d0b8238e4e33c331636'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:59.345465 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-67393c20-9236-4207-a90d-60734021b561 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eadcee324bec4d0b8238e4e33c331636'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:59.357644 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-67393c20-9236-4207-a90d-60734021b561 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eadcee324bec4d0b8238e4e33c331636'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:59.363393 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1293/2583] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:09:59 2022] DELETE /identity/v3/users/eadcee324bec4d0b8238e4e33c331636 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:09:59.625852 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e66e84-e4f5-4999-8894-c0d1ca146dd4 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:59.627714 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-28e66e84-e4f5-4999-8894-c0d1ca146dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:59.641455 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e66e84-e4f5-4999-8894-c0d1ca146dd4 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:59.696552 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aaf9ce-1896-4431-8637-b6ec6d88cf2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:59.697400 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aaf9ce-1896-4431-8637-b6ec6d88cf2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:59.697709 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aaf9ce-1896-4431-8637-b6ec6d88cf2d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:59.697976 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aaf9ce-1896-4431-8637-b6ec6d88cf2d admin admin] PATH_INFO: `/v3/projects/0c8d1b988b6341169af9b3c19fe2a171` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:59.699165 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aaf9ce-1896-4431-8637-b6ec6d88cf2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c8d1b988b6341169af9b3c19fe2a171)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:59.701037 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aaf9ce-1896-4431-8637-b6ec6d88cf2d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:59.746906 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-83aaf9ce-1896-4431-8637-b6ec6d88cf2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c8d1b988b6341169af9b3c19fe2a171', 'request_id': 'req-83aaf9ce-1896-4431-8637-b6ec6d88cf2d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:59.753266 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-83aaf9ce-1896-4431-8637-b6ec6d88cf2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c8d1b988b6341169af9b3c19fe2a171', 'request_id': 'req-83aaf9ce-1896-4431-8637-b6ec6d88cf2d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:59.758501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-83aaf9ce-1896-4431-8637-b6ec6d88cf2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c8d1b988b6341169af9b3c19fe2a171', 'request_id': 'req-83aaf9ce-1896-4431-8637-b6ec6d88cf2d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:59.770084 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-83aaf9ce-1896-4431-8637-b6ec6d88cf2d admin admin] The token cache is being invalidate because project 0c8d1b988b6341169af9b3c19fe2a171 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:09:59.770397 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-83aaf9ce-1896-4431-8637-b6ec6d88cf2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:59.771018 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-83aaf9ce-1896-4431-8637-b6ec6d88cf2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:09:59.772548 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1291/2584] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:09:59 2022] DELETE /identity/v3/projects/0c8d1b988b6341169af9b3c19fe2a171 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:09:59.777971 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4f1f8-3963-4924-ba6e-28e3e827e438 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:59.778297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4f1f8-3963-4924-ba6e-28e3e827e438 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:59.778575 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4f1f8-3963-4924-ba6e-28e3e827e438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:59.796299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-62b4f1f8-3963-4924-ba6e-28e3e827e438 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:59.797476 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-62b4f1f8-3963-4924-ba6e-28e3e827e438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:59.854071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1294/2585] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:59.862342 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c8ab3-7a69-4042-8b5c-3c8408bc0c15 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:59.862651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c8ab3-7a69-4042-8b5c-3c8408bc0c15 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:59.862927 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c8ab3-7a69-4042-8b5c-3c8408bc0c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:59.880972 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-951c8ab3-7a69-4042-8b5c-3c8408bc0c15 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:09:59.882196 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-951c8ab3-7a69-4042-8b5c-3c8408bc0c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:09:59.889961 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1292/2586] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:09:59.894062 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b4f1f8-3963-4924-ba6e-28e3e827e438 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:59.904000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b4f1f8-3963-4924-ba6e-28e3e827e438 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:59.908500 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894bc18-12a1-4cc3-9880-d02d53bfe287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:59.909125 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894bc18-12a1-4cc3-9880-d02d53bfe287 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:59.909406 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894bc18-12a1-4cc3-9880-d02d53bfe287 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:59.909682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894bc18-12a1-4cc3-9880-d02d53bfe287 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:59.910440 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894bc18-12a1-4cc3-9880-d02d53bfe287 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:09:59.910748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894bc18-12a1-4cc3-9880-d02d53bfe287 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:59.912327 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894bc18-12a1-4cc3-9880-d02d53bfe287 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:59.924080 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1295/2587] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:09:59.928102 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c8ab3-7a69-4042-8b5c-3c8408bc0c15 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:59.937114 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c8ab3-7a69-4042-8b5c-3c8408bc0c15 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:09:59.941725 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952eb5b-7f62-4321-86ed-085e5a8ed5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:09:59.942377 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952eb5b-7f62-4321-86ed-085e5a8ed5e5 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:09:59.942655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952eb5b-7f62-4321-86ed-085e5a8ed5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:09:59.942929 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952eb5b-7f62-4321-86ed-085e5a8ed5e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:09:59.943610 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8952eb5b-7f62-4321-86ed-085e5a8ed5e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:09:59.945503 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8952eb5b-7f62-4321-86ed-085e5a8ed5e5 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:09:59.972298 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1293/2588] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:09:59 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:09:59.980230 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894bc18-12a1-4cc3-9880-d02d53bfe287 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:09:59.986940 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894bc18-12a1-4cc3-9880-d02d53bfe287 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:00.040575 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d7feb-eaf7-4154-a674-019631186436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:00.041239 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d7feb-eaf7-4154-a674-019631186436 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:00.041524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d7feb-eaf7-4154-a674-019631186436 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:00.041830 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d7feb-eaf7-4154-a674-019631186436 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:00.042555 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d7feb-eaf7-4154-a674-019631186436 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:00.044538 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d7feb-eaf7-4154-a674-019631186436 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:00.063400 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1296/2589] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:09:59 2022] POST /identity/v3/users => generated 287 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:10:00.067384 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952eb5b-7f62-4321-86ed-085e5a8ed5e5 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:00.077877 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952eb5b-7f62-4321-86ed-085e5a8ed5e5 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:00.082679 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04379d-41f0-4628-bf36-1619411a81ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:00.083356 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04379d-41f0-4628-bf36-1619411a81ac admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:00.083639 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04379d-41f0-4628-bf36-1619411a81ac admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:00.083919 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04379d-41f0-4628-bf36-1619411a81ac admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:00.084672 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04379d-41f0-4628-bf36-1619411a81ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:00.086266 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04379d-41f0-4628-bf36-1619411a81ac admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:00.097619 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1294/2590] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:10:00.101541 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d7feb-eaf7-4154-a674-019631186436 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:00.111836 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d7feb-eaf7-4154-a674-019631186436 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:00.116445 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c85d6-1cc3-45fb-bbec-fb7b0181e34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:00.117091 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c85d6-1cc3-45fb-bbec-fb7b0181e34d admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:00.117375 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c85d6-1cc3-45fb-bbec-fb7b0181e34d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:00.117651 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c85d6-1cc3-45fb-bbec-fb7b0181e34d admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:00.118428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c85d6-1cc3-45fb-bbec-fb7b0181e34d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:00.120010 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c85d6-1cc3-45fb-bbec-fb7b0181e34d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:00.129604 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1297/2591] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:10:00.133510 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04379d-41f0-4628-bf36-1619411a81ac admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:00.143919 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04379d-41f0-4628-bf36-1619411a81ac admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:00.148597 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b3275-fe2f-4f07-a925-e3e778ca1006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:00.150293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b3275-fe2f-4f07-a925-e3e778ca1006 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:00.150607 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b3275-fe2f-4f07-a925-e3e778ca1006 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:00.150882 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b3275-fe2f-4f07-a925-e3e778ca1006 admin admin] PATH_INFO: `/v3/projects/f99e7d13264c4801bbd78669d85fd6ce/users/8f0d8c487e3a4570abea4ad26bcaa34a/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:00.164956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b3275-fe2f-4f07-a925-e3e778ca1006 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f99e7d13264c4801bbd78669d85fd6ce, user_id=8f0d8c487e3a4570abea4ad26bcaa34a, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:00.167665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b3275-fe2f-4f07-a925-e3e778ca1006 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:00.181349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1295/2592] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:10:00 2022] PUT /identity/v3/projects/f99e7d13264c4801bbd78669d85fd6ce/users/8f0d8c487e3a4570abea4ad26bcaa34a/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:00.186488 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab0791-ab89-4ed2-88a6-d6bf85ff27ea None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:00.186777 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab0791-ab89-4ed2-88a6-d6bf85ff27ea None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:00.187056 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab0791-ab89-4ed2-88a6-d6bf85ff27ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:00.204706 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-a5ab0791-ab89-4ed2-88a6-d6bf85ff27ea None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:00.205887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a5ab0791-ab89-4ed2-88a6-d6bf85ff27ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:00.261710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1298/2593] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:10:00.271079 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b3275-fe2f-4f07-a925-e3e778ca1006 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:00.272756 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-fb9b3275-fe2f-4f07-a925-e3e778ca1006 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:00.327377 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b3275-fe2f-4f07-a925-e3e778ca1006 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:00.331001 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715b495-70a7-4098-9980-baec25123466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:00.331632 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715b495-70a7-4098-9980-baec25123466 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:00.331910 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715b495-70a7-4098-9980-baec25123466 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:00.332181 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715b495-70a7-4098-9980-baec25123466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:00.341483 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715b495-70a7-4098-9980-baec25123466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:00.342923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715b495-70a7-4098-9980-baec25123466 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:00.352016 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1296/2594] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:10:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:10:00.934614 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e7d48-0165-4f81-883d-85f63bd3da76 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:00.934985 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e7d48-0165-4f81-883d-85f63bd3da76 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:00.935331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e7d48-0165-4f81-883d-85f63bd3da76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:00.961063 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-fb5e7d48-0165-4f81-883d-85f63bd3da76 None None] MFA Rules not processed for user `8f0d8c487e3a4570abea4ad26bcaa34a`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:00.962415 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-fb5e7d48-0165-4f81-883d-85f63bd3da76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:01.028669 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1299/2595] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:10:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:10:01.039522 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715b495-70a7-4098-9980-baec25123466 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:01.041489 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7715b495-70a7-4098-9980-baec25123466 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:01.095914 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715b495-70a7-4098-9980-baec25123466 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:01.099613 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a0072-1b59-42b5-af8a-4c53d287ab4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:01.100252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a0072-1b59-42b5-af8a-4c53d287ab4a service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:01.100531 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a0072-1b59-42b5-af8a-4c53d287ab4a service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:01.100798 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a0072-1b59-42b5-af8a-4c53d287ab4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:01.110678 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a0072-1b59-42b5-af8a-4c53d287ab4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:01.112100 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a0072-1b59-42b5-af8a-4c53d287ab4a service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:01.121194 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1297/2596] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:10:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:10:05.594689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37db8fe-0cf2-4a1c-88f7-e39b067021b6 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:05.595213 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37db8fe-0cf2-4a1c-88f7-e39b067021b6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:05.595311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37db8fe-0cf2-4a1c-88f7-e39b067021b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:05.613592 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e37db8fe-0cf2-4a1c-88f7-e39b067021b6 None None] MFA Rules not processed for user `8f0d8c487e3a4570abea4ad26bcaa34a`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:05.614862 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e37db8fe-0cf2-4a1c-88f7-e39b067021b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:05.624943 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1300/2597] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:10:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:10:16.719230 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a0072-1b59-42b5-af8a-4c53d287ab4a service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:16.721286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7f6a0072-1b59-42b5-af8a-4c53d287ab4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:16.735699 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a0072-1b59-42b5-af8a-4c53d287ab4a service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:16.740616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376e975-a09b-4da1-b080-8f8da5eca299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:16.741948 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376e975-a09b-4da1-b080-8f8da5eca299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:16.742738 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376e975-a09b-4da1-b080-8f8da5eca299 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:16.743268 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376e975-a09b-4da1-b080-8f8da5eca299 admin admin] PATH_INFO: `/v3/users/5dc01926da9e47f39c4eb0d101831d40` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:16.744495 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376e975-a09b-4da1-b080-8f8da5eca299 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dc01926da9e47f39c4eb0d101831d40)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:16.746444 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376e975-a09b-4da1-b080-8f8da5eca299 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:16.798557 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4376e975-a09b-4da1-b080-8f8da5eca299 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dc01926da9e47f39c4eb0d101831d40'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:16.808847 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4376e975-a09b-4da1-b080-8f8da5eca299 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dc01926da9e47f39c4eb0d101831d40'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:16.810286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4376e975-a09b-4da1-b080-8f8da5eca299 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dc01926da9e47f39c4eb0d101831d40'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:16.818848 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4376e975-a09b-4da1-b080-8f8da5eca299 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dc01926da9e47f39c4eb0d101831d40'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:16.819543 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4376e975-a09b-4da1-b080-8f8da5eca299 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dc01926da9e47f39c4eb0d101831d40'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:16.844638 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1298/2598] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:10:16 2022] DELETE /identity/v3/users/5dc01926da9e47f39c4eb0d101831d40 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:16.849227 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c5f17-243f-4d7d-9a75-ee1f12a64190 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:16.850097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c5f17-243f-4d7d-9a75-ee1f12a64190 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:16.850446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c5f17-243f-4d7d-9a75-ee1f12a64190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:16.869355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-a99c5f17-243f-4d7d-9a75-ee1f12a64190 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:16.870876 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a99c5f17-243f-4d7d-9a75-ee1f12a64190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:16.939970 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1301/2599] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:10:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:10:16.950399 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376e975-a09b-4da1-b080-8f8da5eca299 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:16.952163 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-4376e975-a09b-4da1-b080-8f8da5eca299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:17.011065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376e975-a09b-4da1-b080-8f8da5eca299 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:17.016556 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63d788-f87d-461c-ac3b-8a765b7f1ec7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:17.017216 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63d788-f87d-461c-ac3b-8a765b7f1ec7 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:17.017494 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63d788-f87d-461c-ac3b-8a765b7f1ec7 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:17.017769 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63d788-f87d-461c-ac3b-8a765b7f1ec7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:17.029503 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63d788-f87d-461c-ac3b-8a765b7f1ec7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:17.031476 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63d788-f87d-461c-ac3b-8a765b7f1ec7 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:17.043687 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1299/2600] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:10:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:10:17.373133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c5f17-243f-4d7d-9a75-ee1f12a64190 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:17.374584 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a99c5f17-243f-4d7d-9a75-ee1f12a64190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:17.386927 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c5f17-243f-4d7d-9a75-ee1f12a64190 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:17.391427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc90c6-6427-4c60-9c07-005e6c2fb6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:17.392279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc90c6-6427-4c60-9c07-005e6c2fb6b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:17.392557 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc90c6-6427-4c60-9c07-005e6c2fb6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:17.392832 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc90c6-6427-4c60-9c07-005e6c2fb6b1 admin admin] PATH_INFO: `/v3/projects/61255ec0178640faa3828048625af777` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:17.394182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc90c6-6427-4c60-9c07-005e6c2fb6b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61255ec0178640faa3828048625af777)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:17.396178 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc90c6-6427-4c60-9c07-005e6c2fb6b1 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:17.441339 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b6cc90c6-6427-4c60-9c07-005e6c2fb6b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61255ec0178640faa3828048625af777', 'request_id': 'req-b6cc90c6-6427-4c60-9c07-005e6c2fb6b1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:17.447027 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b6cc90c6-6427-4c60-9c07-005e6c2fb6b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61255ec0178640faa3828048625af777', 'request_id': 'req-b6cc90c6-6427-4c60-9c07-005e6c2fb6b1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:17.452898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b6cc90c6-6427-4c60-9c07-005e6c2fb6b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61255ec0178640faa3828048625af777', 'request_id': 'req-b6cc90c6-6427-4c60-9c07-005e6c2fb6b1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:17.465071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b6cc90c6-6427-4c60-9c07-005e6c2fb6b1 admin admin] The token cache is being invalidate because project 61255ec0178640faa3828048625af777 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:10:17.465361 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b6cc90c6-6427-4c60-9c07-005e6c2fb6b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:17.466003 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b6cc90c6-6427-4c60-9c07-005e6c2fb6b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:17.467858 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1302/2601] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:10:17 2022] DELETE /identity/v3/projects/61255ec0178640faa3828048625af777 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:17.471709 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63d788-f87d-461c-ac3b-8a765b7f1ec7 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:17.481661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63d788-f87d-461c-ac3b-8a765b7f1ec7 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:17.534682 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c8534-2a37-44b9-a070-a429d90b25e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:17.535535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c8534-2a37-44b9-a070-a429d90b25e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:17.535813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c8534-2a37-44b9-a070-a429d90b25e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:17.536085 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c8534-2a37-44b9-a070-a429d90b25e9 admin admin] PATH_INFO: `/v3/users/c5be690dbffc4b57b8210bb4a41e97c8` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:17.537243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c8534-2a37-44b9-a070-a429d90b25e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5be690dbffc4b57b8210bb4a41e97c8)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:17.539194 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c8534-2a37-44b9-a070-a429d90b25e9 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:17.583652 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-958c8534-2a37-44b9-a070-a429d90b25e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5be690dbffc4b57b8210bb4a41e97c8'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:17.590730 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-958c8534-2a37-44b9-a070-a429d90b25e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5be690dbffc4b57b8210bb4a41e97c8'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:17.591369 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-958c8534-2a37-44b9-a070-a429d90b25e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5be690dbffc4b57b8210bb4a41e97c8'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:17.598246 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-958c8534-2a37-44b9-a070-a429d90b25e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5be690dbffc4b57b8210bb4a41e97c8'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:17.598847 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-958c8534-2a37-44b9-a070-a429d90b25e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5be690dbffc4b57b8210bb4a41e97c8'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:17.615674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1300/2602] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:10:17 2022] DELETE /identity/v3/users/c5be690dbffc4b57b8210bb4a41e97c8 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:17.892471 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc90c6-6427-4c60-9c07-005e6c2fb6b1 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:17.894028 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b6cc90c6-6427-4c60-9c07-005e6c2fb6b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:17.908332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc90c6-6427-4c60-9c07-005e6c2fb6b1 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:17.972739 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57b4c9-95f6-4d15-ba96-445b566cb1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:17.973724 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57b4c9-95f6-4d15-ba96-445b566cb1e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:17.974105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57b4c9-95f6-4d15-ba96-445b566cb1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:17.974506 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57b4c9-95f6-4d15-ba96-445b566cb1e7 admin admin] PATH_INFO: `/v3/projects/663748103e0d422caca27dbd8befcc99` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:17.975789 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e57b4c9-95f6-4d15-ba96-445b566cb1e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=663748103e0d422caca27dbd8befcc99)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:17.977851 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e57b4c9-95f6-4d15-ba96-445b566cb1e7 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:18.026418 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2e57b4c9-95f6-4d15-ba96-445b566cb1e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '663748103e0d422caca27dbd8befcc99', 'request_id': 'req-2e57b4c9-95f6-4d15-ba96-445b566cb1e7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:18.031842 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2e57b4c9-95f6-4d15-ba96-445b566cb1e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '663748103e0d422caca27dbd8befcc99', 'request_id': 'req-2e57b4c9-95f6-4d15-ba96-445b566cb1e7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:18.038214 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2e57b4c9-95f6-4d15-ba96-445b566cb1e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '663748103e0d422caca27dbd8befcc99', 'request_id': 'req-2e57b4c9-95f6-4d15-ba96-445b566cb1e7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:18.051133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2e57b4c9-95f6-4d15-ba96-445b566cb1e7 admin admin] The token cache is being invalidate because project 663748103e0d422caca27dbd8befcc99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:10:18.051545 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2e57b4c9-95f6-4d15-ba96-445b566cb1e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:18.052171 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2e57b4c9-95f6-4d15-ba96-445b566cb1e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:18.053740 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1303/2603] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:10:17 2022] DELETE /identity/v3/projects/663748103e0d422caca27dbd8befcc99 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:18.059393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d776e9c-09ef-4f7e-873f-db75feb0e71d None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:18.059891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d776e9c-09ef-4f7e-873f-db75feb0e71d None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:18.060113 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d776e9c-09ef-4f7e-873f-db75feb0e71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:18.078795 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-2d776e9c-09ef-4f7e-873f-db75feb0e71d None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:18.080645 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2d776e9c-09ef-4f7e-873f-db75feb0e71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:18.140444 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1301/2604] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:10:18.148522 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7500e66-c1af-4b90-b896-ef9bf7b5da8c None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:18.148804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7500e66-c1af-4b90-b896-ef9bf7b5da8c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:18.149082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7500e66-c1af-4b90-b896-ef9bf7b5da8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:18.167411 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c7500e66-c1af-4b90-b896-ef9bf7b5da8c None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:18.168799 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c7500e66-c1af-4b90-b896-ef9bf7b5da8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:18.177782 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1304/2605] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:10:18.182791 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d776e9c-09ef-4f7e-873f-db75feb0e71d None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:18.192444 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d776e9c-09ef-4f7e-873f-db75feb0e71d None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:18.197129 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ab82d-6b69-403c-83f5-cc048d7fdda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:18.197777 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ab82d-6b69-403c-83f5-cc048d7fdda7 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:18.198072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ab82d-6b69-403c-83f5-cc048d7fdda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:18.198517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ab82d-6b69-403c-83f5-cc048d7fdda7 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:18.199291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ab82d-6b69-403c-83f5-cc048d7fdda7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:10:18.199591 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ab82d-6b69-403c-83f5-cc048d7fdda7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:18.201630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ab82d-6b69-403c-83f5-cc048d7fdda7 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:18.214830 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1302/2606] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:10:18.219342 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7500e66-c1af-4b90-b896-ef9bf7b5da8c None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:18.229939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7500e66-c1af-4b90-b896-ef9bf7b5da8c None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:18.235819 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f49e2-e141-4b84-8e43-7c0f3d1dc015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:18.236610 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f49e2-e141-4b84-8e43-7c0f3d1dc015 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:18.236902 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f49e2-e141-4b84-8e43-7c0f3d1dc015 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:18.237180 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f49e2-e141-4b84-8e43-7c0f3d1dc015 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:18.237877 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f49e2-e141-4b84-8e43-7c0f3d1dc015 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:18.239824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f49e2-e141-4b84-8e43-7c0f3d1dc015 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:18.272668 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1305/2607] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:10:18 2022] POST /identity/v3/projects => generated 355 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:10:18.276898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ab82d-6b69-403c-83f5-cc048d7fdda7 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:18.286989 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ab82d-6b69-403c-83f5-cc048d7fdda7 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:18.372325 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68214881-1781-4c89-b722-712f7b39b2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:18.372989 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68214881-1781-4c89-b722-712f7b39b2d9 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:18.373290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68214881-1781-4c89-b722-712f7b39b2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:18.373619 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68214881-1781-4c89-b722-712f7b39b2d9 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:18.374349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68214881-1781-4c89-b722-712f7b39b2d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:18.376256 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68214881-1781-4c89-b722-712f7b39b2d9 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:18.396650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1303/2608] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:10:18 2022] POST /identity/v3/users => generated 278 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:10:18.401355 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f49e2-e141-4b84-8e43-7c0f3d1dc015 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:18.411641 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f49e2-e141-4b84-8e43-7c0f3d1dc015 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:18.417056 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91145113-1e0d-4214-8d14-16fec23ef048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:18.417708 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91145113-1e0d-4214-8d14-16fec23ef048 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:18.417984 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91145113-1e0d-4214-8d14-16fec23ef048 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:18.418297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91145113-1e0d-4214-8d14-16fec23ef048 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:18.419009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91145113-1e0d-4214-8d14-16fec23ef048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:18.420590 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91145113-1e0d-4214-8d14-16fec23ef048 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:18.434350 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1306/2609] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:10:18.438061 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68214881-1781-4c89-b722-712f7b39b2d9 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:18.448607 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68214881-1781-4c89-b722-712f7b39b2d9 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:18.453205 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae421e-be23-4a9d-b114-02def11f2a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:18.453859 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae421e-be23-4a9d-b114-02def11f2a9e admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:18.454171 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae421e-be23-4a9d-b114-02def11f2a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:18.454448 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae421e-be23-4a9d-b114-02def11f2a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:18.455190 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae421e-be23-4a9d-b114-02def11f2a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:18.456779 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae421e-be23-4a9d-b114-02def11f2a9e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:18.467921 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1304/2610] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:10:18.478592 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91145113-1e0d-4214-8d14-16fec23ef048 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:18.485405 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91145113-1e0d-4214-8d14-16fec23ef048 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:18.487647 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097ac9a-2136-4d3e-9376-5349eb8fa285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:18.488524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097ac9a-2136-4d3e-9376-5349eb8fa285 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:18.488804 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097ac9a-2136-4d3e-9376-5349eb8fa285 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:18.489078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097ac9a-2136-4d3e-9376-5349eb8fa285 admin admin] PATH_INFO: `/v3/projects/6155c372999642c18bf9408844485ced/users/43d1ebb273324d3bac99d17f866fcede/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:18.504425 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097ac9a-2136-4d3e-9376-5349eb8fa285 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6155c372999642c18bf9408844485ced, user_id=43d1ebb273324d3bac99d17f866fcede, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:18.507098 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097ac9a-2136-4d3e-9376-5349eb8fa285 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:18.521299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1307/2611] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:10:18 2022] PUT /identity/v3/projects/6155c372999642c18bf9408844485ced/users/43d1ebb273324d3bac99d17f866fcede/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:10:18.532401 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b1f44-d6d6-4b15-b737-60023d2f5179 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:18.532686 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b1f44-d6d6-4b15-b737-60023d2f5179 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:18.532962 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b1f44-d6d6-4b15-b737-60023d2f5179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:18.559655 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-0d6b1f44-d6d6-4b15-b737-60023d2f5179 None None] MFA Rules not processed for user `43d1ebb273324d3bac99d17f866fcede`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:18.561105 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0d6b1f44-d6d6-4b15-b737-60023d2f5179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:18.626663 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1305/2612] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:10:18 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:10:18.630885 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097ac9a-2136-4d3e-9376-5349eb8fa285 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:18.691923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097ac9a-2136-4d3e-9376-5349eb8fa285 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:18.696806 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e29b919-d3b4-485e-a132-ea7b8eb5e2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:18.697473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29b919-d3b4-485e-a132-ea7b8eb5e2ea admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:18.697762 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29b919-d3b4-485e-a132-ea7b8eb5e2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:18.698099 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29b919-d3b4-485e-a132-ea7b8eb5e2ea admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:18.698826 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e29b919-d3b4-485e-a132-ea7b8eb5e2ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:18.700736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e29b919-d3b4-485e-a132-ea7b8eb5e2ea admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:18.736292 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1308/2613] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:10:18 2022] POST /identity/v3/projects => generated 355 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:10:18.737365 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b1f44-d6d6-4b15-b737-60023d2f5179 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:18.748189 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b1f44-d6d6-4b15-b737-60023d2f5179 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:18.802723 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85bc14-a690-4554-bfeb-f9f7923b1b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:18.803387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85bc14-a690-4554-bfeb-f9f7923b1b6d admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:18.803672 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85bc14-a690-4554-bfeb-f9f7923b1b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:18.803944 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85bc14-a690-4554-bfeb-f9f7923b1b6d admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:18.804623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85bc14-a690-4554-bfeb-f9f7923b1b6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:18.806597 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85bc14-a690-4554-bfeb-f9f7923b1b6d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:18.825831 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1306/2614] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:10:18 2022] POST /identity/v3/users => generated 284 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:10:18.830103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e29b919-d3b4-485e-a132-ea7b8eb5e2ea admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:18.840770 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e29b919-d3b4-485e-a132-ea7b8eb5e2ea admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:18.846588 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f115522-12ed-4826-b080-b411501be0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:18.847368 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f115522-12ed-4826-b080-b411501be0be admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:18.847729 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f115522-12ed-4826-b080-b411501be0be admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:18.848105 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f115522-12ed-4826-b080-b411501be0be admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:18.848931 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f115522-12ed-4826-b080-b411501be0be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:18.850664 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f115522-12ed-4826-b080-b411501be0be admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:18.862684 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1309/2615] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:10:18.866901 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85bc14-a690-4554-bfeb-f9f7923b1b6d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:18.876557 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85bc14-a690-4554-bfeb-f9f7923b1b6d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:18.881044 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa0152-81e7-41ce-b0e8-82db1dcde6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:18.881892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa0152-81e7-41ce-b0e8-82db1dcde6e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:18.882212 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa0152-81e7-41ce-b0e8-82db1dcde6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:18.882474 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa0152-81e7-41ce-b0e8-82db1dcde6e1 admin admin] PATH_INFO: `/v3/projects/7f37f33d0d834d38bd824e47e5b4886b/users/5a055c9f1f7f4899a53cc147b0eb2da5/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:18.897100 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa0152-81e7-41ce-b0e8-82db1dcde6e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f37f33d0d834d38bd824e47e5b4886b, user_id=5a055c9f1f7f4899a53cc147b0eb2da5, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:18.900380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa0152-81e7-41ce-b0e8-82db1dcde6e1 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:18.913560 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1307/2616] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:10:18 2022] PUT /identity/v3/projects/7f37f33d0d834d38bd824e47e5b4886b/users/5a055c9f1f7f4899a53cc147b0eb2da5/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:18.917555 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f115522-12ed-4826-b080-b411501be0be admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:18.927321 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f115522-12ed-4826-b080-b411501be0be admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:18.979932 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1046b-47dc-4103-8070-c59ef0596fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:18.980835 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1046b-47dc-4103-8070-c59ef0596fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:18.981120 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1046b-47dc-4103-8070-c59ef0596fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:18.981390 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1046b-47dc-4103-8070-c59ef0596fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:18.982089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1046b-47dc-4103-8070-c59ef0596fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:18.983723 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1046b-47dc-4103-8070-c59ef0596fc4 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:18.993335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1310/2617] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:10:18.997519 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa0152-81e7-41ce-b0e8-82db1dcde6e1 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:19.007547 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa0152-81e7-41ce-b0e8-82db1dcde6e1 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:19.012578 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca725f7-bbf5-4445-9621-349a28148ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:19.013437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca725f7-bbf5-4445-9621-349a28148ab8 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:19.013712 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca725f7-bbf5-4445-9621-349a28148ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:19.013982 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca725f7-bbf5-4445-9621-349a28148ab8 admin admin] PATH_INFO: `/v3/projects/7f37f33d0d834d38bd824e47e5b4886b/users/5a055c9f1f7f4899a53cc147b0eb2da5/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:19.015956 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca725f7-bbf5-4445-9621-349a28148ab8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f37f33d0d834d38bd824e47e5b4886b, user_id=5a055c9f1f7f4899a53cc147b0eb2da5, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:19.018614 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca725f7-bbf5-4445-9621-349a28148ab8 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:19.028914 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1308/2618] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:10:18 2022] PUT /identity/v3/projects/7f37f33d0d834d38bd824e47e5b4886b/users/5a055c9f1f7f4899a53cc147b0eb2da5/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:10:19.039519 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f124067-0d22-4410-a61b-32fe2ed19f95 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:19.039810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f124067-0d22-4410-a61b-32fe2ed19f95 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:19.040117 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f124067-0d22-4410-a61b-32fe2ed19f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:19.065604 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-6f124067-0d22-4410-a61b-32fe2ed19f95 None None] MFA Rules not processed for user `5a055c9f1f7f4899a53cc147b0eb2da5`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:19.066833 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6f124067-0d22-4410-a61b-32fe2ed19f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:19.133626 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1311/2619] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:10:19 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:10:19.144774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca725f7-bbf5-4445-9621-349a28148ab8 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:19.146493 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-bca725f7-bbf5-4445-9621-349a28148ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:19.201670 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca725f7-bbf5-4445-9621-349a28148ab8 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:19.205301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c07c7-dfa1-4fcc-b9e7-6fff2fc2bf39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:19.205924 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c07c7-dfa1-4fcc-b9e7-6fff2fc2bf39 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:19.206237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c07c7-dfa1-4fcc-b9e7-6fff2fc2bf39 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:19.206508 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c07c7-dfa1-4fcc-b9e7-6fff2fc2bf39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:19.217104 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422c07c7-dfa1-4fcc-b9e7-6fff2fc2bf39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:19.218585 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422c07c7-dfa1-4fcc-b9e7-6fff2fc2bf39 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:19.228686 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1309/2620] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:10:19 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:10:19.298960 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f124067-0d22-4410-a61b-32fe2ed19f95 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:19.308229 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f124067-0d22-4410-a61b-32fe2ed19f95 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:19.311789 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e8970-9b63-4130-8f6a-4ce26e12d9ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:19.312408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e8970-9b63-4130-8f6a-4ce26e12d9ff service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:19.312685 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e8970-9b63-4130-8f6a-4ce26e12d9ff service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:19.312953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e8970-9b63-4130-8f6a-4ce26e12d9ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:19.366702 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e8970-9b63-4130-8f6a-4ce26e12d9ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:19.368114 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e8970-9b63-4130-8f6a-4ce26e12d9ff service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:19.377018 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1312/2621] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:10:19 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:10:20.688011 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c07c7-dfa1-4fcc-b9e7-6fff2fc2bf39 service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:20.689545 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-422c07c7-dfa1-4fcc-b9e7-6fff2fc2bf39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:20.757343 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c07c7-dfa1-4fcc-b9e7-6fff2fc2bf39 service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:20.762928 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6debb8-cb95-442a-9de7-028ed8d064d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf9bc2fa959d4706b9f9e87509829059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:20.763810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6debb8-cb95-442a-9de7-028ed8d064d1 service placement] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:20.764090 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6debb8-cb95-442a-9de7-028ed8d064d1 service placement] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:20.764713 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6debb8-cb95-442a-9de7-028ed8d064d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:20.766519 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ab6debb8-cb95-442a-9de7-028ed8d064d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:20.778301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6debb8-cb95-442a-9de7-028ed8d064d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:20.779878 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6debb8-cb95-442a-9de7-028ed8d064d1 service placement] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:20.790343 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1310/2622] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:10:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:10:22.095124 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45255e-08ca-4596-a8b4-bf099212f4b9 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:22.095580 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45255e-08ca-4596-a8b4-bf099212f4b9 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:22.095700 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45255e-08ca-4596-a8b4-bf099212f4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:22.113009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3b45255e-08ca-4596-a8b4-bf099212f4b9 None None] MFA Rules not processed for user `8f0d8c487e3a4570abea4ad26bcaa34a`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:22.114230 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3b45255e-08ca-4596-a8b4-bf099212f4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:22.169467 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1313/2623] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:10:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:10:22.640827 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6debb8-cb95-442a-9de7-028ed8d064d1 service placement] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:22.642238 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ab6debb8-cb95-442a-9de7-028ed8d064d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:22.694644 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6debb8-cb95-442a-9de7-028ed8d064d1 service placement] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:22.698728 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f9195-5d96-450c-add5-82e02bf2a195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:22.699360 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f9195-5d96-450c-add5-82e02bf2a195 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:22.699639 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f9195-5d96-450c-add5-82e02bf2a195 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:22.699911 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f9195-5d96-450c-add5-82e02bf2a195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:22.701623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-171f9195-5d96-450c-add5-82e02bf2a195 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:22.710727 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f9195-5d96-450c-add5-82e02bf2a195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:22.712706 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f9195-5d96-450c-add5-82e02bf2a195 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:22.720400 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1311/2624] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:10:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:10:34.081272 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b45255e-08ca-4596-a8b4-bf099212f4b9 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:34.142971 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b45255e-08ca-4596-a8b4-bf099212f4b9 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:34.147367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce14cf-b935-4c7b-bc6c-2de316590b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:34.148211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce14cf-b935-4c7b-bc6c-2de316590b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:34.148486 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce14cf-b935-4c7b-bc6c-2de316590b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:34.148756 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce14cf-b935-4c7b-bc6c-2de316590b0c admin admin] PATH_INFO: `/v3/users/43d1ebb273324d3bac99d17f866fcede` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:34.149908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce14cf-b935-4c7b-bc6c-2de316590b0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=43d1ebb273324d3bac99d17f866fcede)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:34.151861 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce14cf-b935-4c7b-bc6c-2de316590b0c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:34.197318 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a8ce14cf-b935-4c7b-bc6c-2de316590b0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43d1ebb273324d3bac99d17f866fcede'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:34.204448 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a8ce14cf-b935-4c7b-bc6c-2de316590b0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43d1ebb273324d3bac99d17f866fcede'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:34.211462 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a8ce14cf-b935-4c7b-bc6c-2de316590b0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43d1ebb273324d3bac99d17f866fcede'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:34.212074 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a8ce14cf-b935-4c7b-bc6c-2de316590b0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43d1ebb273324d3bac99d17f866fcede'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:34.223398 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a8ce14cf-b935-4c7b-bc6c-2de316590b0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43d1ebb273324d3bac99d17f866fcede'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:34.229348 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1314/2625] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:10:34 2022] DELETE /identity/v3/users/43d1ebb273324d3bac99d17f866fcede => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:34.234062 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc46d7-4e21-48ed-8a49-ee00654e6303 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:34.234547 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc46d7-4e21-48ed-8a49-ee00654e6303 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:34.234665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc46d7-4e21-48ed-8a49-ee00654e6303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:34.251691 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-96cc46d7-4e21-48ed-8a49-ee00654e6303 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:34.252863 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-96cc46d7-4e21-48ed-8a49-ee00654e6303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:34.307447 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1312/2626] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:10:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:10:34.317648 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce14cf-b935-4c7b-bc6c-2de316590b0c admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:34.319128 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a8ce14cf-b935-4c7b-bc6c-2de316590b0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:34.375724 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce14cf-b935-4c7b-bc6c-2de316590b0c admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:34.379296 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad2328-3550-4874-bf18-434950c3324b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:34.379980 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad2328-3550-4874-bf18-434950c3324b service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:34.380255 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad2328-3550-4874-bf18-434950c3324b service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:34.380520 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad2328-3550-4874-bf18-434950c3324b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:34.389835 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad2328-3550-4874-bf18-434950c3324b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:34.391302 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad2328-3550-4874-bf18-434950c3324b service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:34.400327 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1315/2627] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:10:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:10:34.662738 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc46d7-4e21-48ed-8a49-ee00654e6303 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:34.664592 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-96cc46d7-4e21-48ed-8a49-ee00654e6303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:34.677282 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc46d7-4e21-48ed-8a49-ee00654e6303 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:34.681967 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147548f-55b1-4143-83cc-cdf3a13c1137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:34.683492 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147548f-55b1-4143-83cc-cdf3a13c1137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:34.683847 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147548f-55b1-4143-83cc-cdf3a13c1137 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:34.684127 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147548f-55b1-4143-83cc-cdf3a13c1137 admin admin] PATH_INFO: `/v3/projects/6155c372999642c18bf9408844485ced` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:34.686031 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147548f-55b1-4143-83cc-cdf3a13c1137 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6155c372999642c18bf9408844485ced)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:34.688053 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147548f-55b1-4143-83cc-cdf3a13c1137 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:34.734333 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e147548f-55b1-4143-83cc-cdf3a13c1137 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6155c372999642c18bf9408844485ced', 'request_id': 'req-e147548f-55b1-4143-83cc-cdf3a13c1137'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:34.740267 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e147548f-55b1-4143-83cc-cdf3a13c1137 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6155c372999642c18bf9408844485ced', 'request_id': 'req-e147548f-55b1-4143-83cc-cdf3a13c1137'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:34.746666 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e147548f-55b1-4143-83cc-cdf3a13c1137 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6155c372999642c18bf9408844485ced', 'request_id': 'req-e147548f-55b1-4143-83cc-cdf3a13c1137'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:34.758292 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e147548f-55b1-4143-83cc-cdf3a13c1137 admin admin] The token cache is being invalidate because project 6155c372999642c18bf9408844485ced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:10:34.758574 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e147548f-55b1-4143-83cc-cdf3a13c1137 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:34.759182 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e147548f-55b1-4143-83cc-cdf3a13c1137 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:34.760684 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1313/2628] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:10:34 2022] DELETE /identity/v3/projects/6155c372999642c18bf9408844485ced => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:34.764678 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad2328-3550-4874-bf18-434950c3324b service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:34.774064 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad2328-3550-4874-bf18-434950c3324b service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:34.826865 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02cdc1-631d-45c2-8821-e29ad05561e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:34.827718 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02cdc1-631d-45c2-8821-e29ad05561e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:34.827992 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02cdc1-631d-45c2-8821-e29ad05561e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:34.828264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02cdc1-631d-45c2-8821-e29ad05561e4 admin admin] PATH_INFO: `/v3/users/5a055c9f1f7f4899a53cc147b0eb2da5` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:34.829421 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02cdc1-631d-45c2-8821-e29ad05561e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a055c9f1f7f4899a53cc147b0eb2da5)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:34.831325 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02cdc1-631d-45c2-8821-e29ad05561e4 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:34.875459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9c02cdc1-631d-45c2-8821-e29ad05561e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a055c9f1f7f4899a53cc147b0eb2da5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:34.881901 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9c02cdc1-631d-45c2-8821-e29ad05561e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a055c9f1f7f4899a53cc147b0eb2da5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:34.890017 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9c02cdc1-631d-45c2-8821-e29ad05561e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a055c9f1f7f4899a53cc147b0eb2da5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:34.891711 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9c02cdc1-631d-45c2-8821-e29ad05561e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a055c9f1f7f4899a53cc147b0eb2da5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:34.903389 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9c02cdc1-631d-45c2-8821-e29ad05561e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a055c9f1f7f4899a53cc147b0eb2da5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:34.909639 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1316/2629] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:10:34 2022] DELETE /identity/v3/users/5a055c9f1f7f4899a53cc147b0eb2da5 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:35.257371 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147548f-55b1-4143-83cc-cdf3a13c1137 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:35.259363 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e147548f-55b1-4143-83cc-cdf3a13c1137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:35.272864 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147548f-55b1-4143-83cc-cdf3a13c1137 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:35.335753 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6ca7f-5507-4398-8161-c1a20133243a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:35.336601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6ca7f-5507-4398-8161-c1a20133243a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:35.336882 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6ca7f-5507-4398-8161-c1a20133243a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:35.337149 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6ca7f-5507-4398-8161-c1a20133243a admin admin] PATH_INFO: `/v3/projects/7f37f33d0d834d38bd824e47e5b4886b` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:35.338334 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6ca7f-5507-4398-8161-c1a20133243a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f37f33d0d834d38bd824e47e5b4886b)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:35.340887 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6ca7f-5507-4398-8161-c1a20133243a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:35.386968 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-adb6ca7f-5507-4398-8161-c1a20133243a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f37f33d0d834d38bd824e47e5b4886b', 'request_id': 'req-adb6ca7f-5507-4398-8161-c1a20133243a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:35.393651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-adb6ca7f-5507-4398-8161-c1a20133243a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f37f33d0d834d38bd824e47e5b4886b', 'request_id': 'req-adb6ca7f-5507-4398-8161-c1a20133243a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:35.399228 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-adb6ca7f-5507-4398-8161-c1a20133243a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f37f33d0d834d38bd824e47e5b4886b', 'request_id': 'req-adb6ca7f-5507-4398-8161-c1a20133243a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:35.412700 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-adb6ca7f-5507-4398-8161-c1a20133243a admin admin] The token cache is being invalidate because project 7f37f33d0d834d38bd824e47e5b4886b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:10:35.413623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-adb6ca7f-5507-4398-8161-c1a20133243a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:35.414427 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-adb6ca7f-5507-4398-8161-c1a20133243a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:35.416280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1314/2630] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:10:35 2022] DELETE /identity/v3/projects/7f37f33d0d834d38bd824e47e5b4886b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:35.421249 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc26e1e-6187-4e72-966b-8821b40850a3 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:35.421717 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc26e1e-6187-4e72-966b-8821b40850a3 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:35.421813 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc26e1e-6187-4e72-966b-8821b40850a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:35.439324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3dc26e1e-6187-4e72-966b-8821b40850a3 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:35.440541 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3dc26e1e-6187-4e72-966b-8821b40850a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:35.498971 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1317/2631] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:10:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:10:35.507045 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96321f-0d60-45da-a2eb-71e509983f69 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:35.507326 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96321f-0d60-45da-a2eb-71e509983f69 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:35.507595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96321f-0d60-45da-a2eb-71e509983f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:35.526043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-3b96321f-0d60-45da-a2eb-71e509983f69 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:35.527868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3b96321f-0d60-45da-a2eb-71e509983f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:35.535860 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1315/2632] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:10:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:10:35.540079 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc26e1e-6187-4e72-966b-8821b40850a3 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:35.549673 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc26e1e-6187-4e72-966b-8821b40850a3 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:35.554497 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ca544-ad40-4b40-8145-f7812f460694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:35.555139 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ca544-ad40-4b40-8145-f7812f460694 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:35.555413 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ca544-ad40-4b40-8145-f7812f460694 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:35.555682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ca544-ad40-4b40-8145-f7812f460694 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:35.556424 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ca544-ad40-4b40-8145-f7812f460694 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:10:35.556726 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ca544-ad40-4b40-8145-f7812f460694 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:35.558358 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ca544-ad40-4b40-8145-f7812f460694 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:35.570631 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1318/2633] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:10:35.575888 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96321f-0d60-45da-a2eb-71e509983f69 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:35.585249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96321f-0d60-45da-a2eb-71e509983f69 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:35.590885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd26b84-4684-42e9-91e9-ac7ddf5d40cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:35.591534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd26b84-4684-42e9-91e9-ac7ddf5d40cc admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:35.591807 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd26b84-4684-42e9-91e9-ac7ddf5d40cc admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:35.592114 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd26b84-4684-42e9-91e9-ac7ddf5d40cc admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:35.592749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd26b84-4684-42e9-91e9-ac7ddf5d40cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:35.594680 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd26b84-4684-42e9-91e9-ac7ddf5d40cc admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:35.624647 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1316/2634] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:10:35 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:10:35.629537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ca544-ad40-4b40-8145-f7812f460694 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:35.642005 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ca544-ad40-4b40-8145-f7812f460694 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:35.697633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17969c0c-ae2a-4bda-bec1-5db05117003b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:35.699083 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17969c0c-ae2a-4bda-bec1-5db05117003b admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:35.699368 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17969c0c-ae2a-4bda-bec1-5db05117003b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:35.699637 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17969c0c-ae2a-4bda-bec1-5db05117003b admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:35.700315 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17969c0c-ae2a-4bda-bec1-5db05117003b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:35.702268 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17969c0c-ae2a-4bda-bec1-5db05117003b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:35.724544 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1319/2635] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:10:35 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:10:35.725830 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd26b84-4684-42e9-91e9-ac7ddf5d40cc admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:35.736187 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd26b84-4684-42e9-91e9-ac7ddf5d40cc admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:35.740839 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac58a20-d64f-43ea-9bcf-97fad7bdb4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:35.741490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac58a20-d64f-43ea-9bcf-97fad7bdb4ad admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:35.741766 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac58a20-d64f-43ea-9bcf-97fad7bdb4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:35.742030 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac58a20-d64f-43ea-9bcf-97fad7bdb4ad admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:35.742784 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac58a20-d64f-43ea-9bcf-97fad7bdb4ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:35.744389 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac58a20-d64f-43ea-9bcf-97fad7bdb4ad admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:35.755193 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1317/2636] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:10:35.759153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17969c0c-ae2a-4bda-bec1-5db05117003b admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:35.769914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17969c0c-ae2a-4bda-bec1-5db05117003b admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:35.774633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd8717-9d41-432a-bedd-dea6cedf4454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:35.775292 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd8717-9d41-432a-bedd-dea6cedf4454 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:35.775572 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd8717-9d41-432a-bedd-dea6cedf4454 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:35.775850 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd8717-9d41-432a-bedd-dea6cedf4454 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:35.776566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd8717-9d41-432a-bedd-dea6cedf4454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:35.778180 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd8717-9d41-432a-bedd-dea6cedf4454 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:35.788397 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1320/2637] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:10:35.792276 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac58a20-d64f-43ea-9bcf-97fad7bdb4ad admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:35.802286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac58a20-d64f-43ea-9bcf-97fad7bdb4ad admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:35.806906 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aae06e-b473-468f-8471-e41b639b3936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:35.808271 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aae06e-b473-468f-8471-e41b639b3936 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:35.808551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aae06e-b473-468f-8471-e41b639b3936 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:35.808823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aae06e-b473-468f-8471-e41b639b3936 admin admin] PATH_INFO: `/v3/projects/418128a55411488d879d49bc4c984c71/users/10370966ccee4cd9872e618ddcc9c23f/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:35.822807 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aae06e-b473-468f-8471-e41b639b3936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=418128a55411488d879d49bc4c984c71, user_id=10370966ccee4cd9872e618ddcc9c23f, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:35.826079 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aae06e-b473-468f-8471-e41b639b3936 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:35.839037 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1318/2638] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:10:35 2022] PUT /identity/v3/projects/418128a55411488d879d49bc4c984c71/users/10370966ccee4cd9872e618ddcc9c23f/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:35.850908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe7074-1da7-402a-8698-616bf07dd217 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:35.851200 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe7074-1da7-402a-8698-616bf07dd217 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:35.851480 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe7074-1da7-402a-8698-616bf07dd217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:35.878026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-93fe7074-1da7-402a-8698-616bf07dd217 None None] MFA Rules not processed for user `10370966ccee4cd9872e618ddcc9c23f`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:35.879233 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-93fe7074-1da7-402a-8698-616bf07dd217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:35.939478 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1321/2639] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:10:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:10:35.943786 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aae06e-b473-468f-8471-e41b639b3936 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:36.002845 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aae06e-b473-468f-8471-e41b639b3936 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:36.007468 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b420a086-3ace-4700-aeac-08d540bdd0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:36.008116 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b420a086-3ace-4700-aeac-08d540bdd0d7 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:36.008396 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b420a086-3ace-4700-aeac-08d540bdd0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:36.008671 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b420a086-3ace-4700-aeac-08d540bdd0d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:36.009356 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b420a086-3ace-4700-aeac-08d540bdd0d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:36.011286 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b420a086-3ace-4700-aeac-08d540bdd0d7 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:36.035230 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1319/2640] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:10:35 2022] POST /identity/v3/projects => generated 387 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:10:36.039264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe7074-1da7-402a-8698-616bf07dd217 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:36.050183 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe7074-1da7-402a-8698-616bf07dd217 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:36.102484 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33a9ff-be4b-44fd-937f-560ddaf65606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:36.103139 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33a9ff-be4b-44fd-937f-560ddaf65606 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:36.103420 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33a9ff-be4b-44fd-937f-560ddaf65606 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:36.103694 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33a9ff-be4b-44fd-937f-560ddaf65606 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:36.104378 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33a9ff-be4b-44fd-937f-560ddaf65606 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:36.107133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33a9ff-be4b-44fd-937f-560ddaf65606 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:36.127179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1322/2641] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:10:36 2022] POST /identity/v3/users => generated 300 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:10:36.131135 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b420a086-3ace-4700-aeac-08d540bdd0d7 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:36.141399 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b420a086-3ace-4700-aeac-08d540bdd0d7 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:36.146088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb249610-3014-4d9e-a852-78dd97f8375c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:36.146764 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb249610-3014-4d9e-a852-78dd97f8375c admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:36.147044 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb249610-3014-4d9e-a852-78dd97f8375c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:36.147314 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb249610-3014-4d9e-a852-78dd97f8375c admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:36.148027 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb249610-3014-4d9e-a852-78dd97f8375c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:36.149596 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb249610-3014-4d9e-a852-78dd97f8375c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:36.159751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1320/2642] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:10:36.163703 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33a9ff-be4b-44fd-937f-560ddaf65606 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:36.173762 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33a9ff-be4b-44fd-937f-560ddaf65606 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:36.178368 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c4e58-77bf-4902-89b2-3689ccd58336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:36.179230 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c4e58-77bf-4902-89b2-3689ccd58336 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:36.179511 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c4e58-77bf-4902-89b2-3689ccd58336 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:36.179812 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c4e58-77bf-4902-89b2-3689ccd58336 admin admin] PATH_INFO: `/v3/projects/28093a9f43fd4573b89980992f5d16d8/users/e58afd783c1341f58487c1bdbe98e1f1/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:36.194349 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c4e58-77bf-4902-89b2-3689ccd58336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28093a9f43fd4573b89980992f5d16d8, user_id=e58afd783c1341f58487c1bdbe98e1f1, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:36.197319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c4e58-77bf-4902-89b2-3689ccd58336 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:36.209541 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1323/2643] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:10:36 2022] PUT /identity/v3/projects/28093a9f43fd4573b89980992f5d16d8/users/e58afd783c1341f58487c1bdbe98e1f1/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:36.213379 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb249610-3014-4d9e-a852-78dd97f8375c admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:36.223124 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb249610-3014-4d9e-a852-78dd97f8375c admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:36.276236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b84e4-8c2d-4e8b-b276-d2fe005a19b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:36.276866 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b84e4-8c2d-4e8b-b276-d2fe005a19b5 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:36.277141 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b84e4-8c2d-4e8b-b276-d2fe005a19b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:36.277410 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b84e4-8c2d-4e8b-b276-d2fe005a19b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:36.278134 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910b84e4-8c2d-4e8b-b276-d2fe005a19b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:36.279702 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910b84e4-8c2d-4e8b-b276-d2fe005a19b5 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:36.289038 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1321/2644] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:10:36.293360 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c4e58-77bf-4902-89b2-3689ccd58336 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:36.303892 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c4e58-77bf-4902-89b2-3689ccd58336 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:36.309069 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355625a-fe26-4d33-9a27-8b5c3b58bd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:36.309937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355625a-fe26-4d33-9a27-8b5c3b58bd8f admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:36.310250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355625a-fe26-4d33-9a27-8b5c3b58bd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:36.310524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355625a-fe26-4d33-9a27-8b5c3b58bd8f admin admin] PATH_INFO: `/v3/projects/28093a9f43fd4573b89980992f5d16d8/users/e58afd783c1341f58487c1bdbe98e1f1/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:36.312534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355625a-fe26-4d33-9a27-8b5c3b58bd8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=28093a9f43fd4573b89980992f5d16d8, user_id=e58afd783c1341f58487c1bdbe98e1f1, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:36.315204 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355625a-fe26-4d33-9a27-8b5c3b58bd8f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:36.325132 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1324/2645] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:10:36 2022] PUT /identity/v3/projects/28093a9f43fd4573b89980992f5d16d8/users/e58afd783c1341f58487c1bdbe98e1f1/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:36.335170 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23b442-456b-4748-b0b7-162c6e7258f2 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:36.335455 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23b442-456b-4748-b0b7-162c6e7258f2 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:36.335728 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23b442-456b-4748-b0b7-162c6e7258f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:36.363798 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ca23b442-456b-4748-b0b7-162c6e7258f2 None None] MFA Rules not processed for user `e58afd783c1341f58487c1bdbe98e1f1`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:36.364996 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ca23b442-456b-4748-b0b7-162c6e7258f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:36.428759 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1322/2646] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:10:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:10:36.439503 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355625a-fe26-4d33-9a27-8b5c3b58bd8f admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:36.441274 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a355625a-fe26-4d33-9a27-8b5c3b58bd8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:36.497754 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355625a-fe26-4d33-9a27-8b5c3b58bd8f admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:36.501340 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4e6d5-781f-42ff-bba7-2a017cf666eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:36.501968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4e6d5-781f-42ff-bba7-2a017cf666eb service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:36.502276 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4e6d5-781f-42ff-bba7-2a017cf666eb service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:36.502554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4e6d5-781f-42ff-bba7-2a017cf666eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:36.511214 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef4e6d5-781f-42ff-bba7-2a017cf666eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:36.512708 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef4e6d5-781f-42ff-bba7-2a017cf666eb service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:36.521755 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1325/2647] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:10:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:10:37.033158 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23b442-456b-4748-b0b7-162c6e7258f2 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:37.093585 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23b442-456b-4748-b0b7-162c6e7258f2 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:37.098069 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d30788-fdd5-46df-8e14-1780e182d628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:37.098946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d30788-fdd5-46df-8e14-1780e182d628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:37.099225 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d30788-fdd5-46df-8e14-1780e182d628 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:37.099498 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d30788-fdd5-46df-8e14-1780e182d628 admin admin] PATH_INFO: `/v3/users/10370966ccee4cd9872e618ddcc9c23f` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:37.100667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d30788-fdd5-46df-8e14-1780e182d628 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10370966ccee4cd9872e618ddcc9c23f)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:37.102661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d30788-fdd5-46df-8e14-1780e182d628 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:37.149848 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-14d30788-fdd5-46df-8e14-1780e182d628 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10370966ccee4cd9872e618ddcc9c23f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:37.156453 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-14d30788-fdd5-46df-8e14-1780e182d628 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10370966ccee4cd9872e618ddcc9c23f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:37.157110 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-14d30788-fdd5-46df-8e14-1780e182d628 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10370966ccee4cd9872e618ddcc9c23f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:37.163764 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-14d30788-fdd5-46df-8e14-1780e182d628 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10370966ccee4cd9872e618ddcc9c23f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:37.164407 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-14d30788-fdd5-46df-8e14-1780e182d628 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10370966ccee4cd9872e618ddcc9c23f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:37.181031 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1323/2648] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:10:37 2022] DELETE /identity/v3/users/10370966ccee4cd9872e618ddcc9c23f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:37.185896 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab5e5b-e447-4633-8e1c-d1bb33518946 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:37.186194 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab5e5b-e447-4633-8e1c-d1bb33518946 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:37.186493 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab5e5b-e447-4633-8e1c-d1bb33518946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:37.204325 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-8cab5e5b-e447-4633-8e1c-d1bb33518946 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:37.205527 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-8cab5e5b-e447-4633-8e1c-d1bb33518946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:37.265137 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1326/2649] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:10:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:10:37.274723 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d30788-fdd5-46df-8e14-1780e182d628 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:37.276129 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-14d30788-fdd5-46df-8e14-1780e182d628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:37.330792 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d30788-fdd5-46df-8e14-1780e182d628 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:37.334391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5d4af-beb5-4987-926f-ad2474163d2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:37.335031 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5d4af-beb5-4987-926f-ad2474163d2e service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:37.335310 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5d4af-beb5-4987-926f-ad2474163d2e service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:37.335583 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5d4af-beb5-4987-926f-ad2474163d2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:37.344569 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f5d4af-beb5-4987-926f-ad2474163d2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:37.345988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f5d4af-beb5-4987-926f-ad2474163d2e service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:37.355801 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1324/2650] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:10:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:10:37.646416 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab5e5b-e447-4633-8e1c-d1bb33518946 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:37.648629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-8cab5e5b-e447-4633-8e1c-d1bb33518946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:37.664457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab5e5b-e447-4633-8e1c-d1bb33518946 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:37.669671 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1de660-92f9-46ab-8d30-59248f940238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:37.670793 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1de660-92f9-46ab-8d30-59248f940238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:37.671075 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1de660-92f9-46ab-8d30-59248f940238 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:37.671348 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1de660-92f9-46ab-8d30-59248f940238 admin admin] PATH_INFO: `/v3/projects/418128a55411488d879d49bc4c984c71` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:37.672555 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1de660-92f9-46ab-8d30-59248f940238 admin admin] RBAC: Authorizing `identity:delete_project(project_id=418128a55411488d879d49bc4c984c71)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:37.674458 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1de660-92f9-46ab-8d30-59248f940238 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:37.723792 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ef1de660-92f9-46ab-8d30-59248f940238 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '418128a55411488d879d49bc4c984c71', 'request_id': 'req-ef1de660-92f9-46ab-8d30-59248f940238'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:37.729336 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ef1de660-92f9-46ab-8d30-59248f940238 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '418128a55411488d879d49bc4c984c71', 'request_id': 'req-ef1de660-92f9-46ab-8d30-59248f940238'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:37.736324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ef1de660-92f9-46ab-8d30-59248f940238 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '418128a55411488d879d49bc4c984c71', 'request_id': 'req-ef1de660-92f9-46ab-8d30-59248f940238'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:37.748901 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ef1de660-92f9-46ab-8d30-59248f940238 admin admin] The token cache is being invalidate because project 418128a55411488d879d49bc4c984c71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:10:37.749187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ef1de660-92f9-46ab-8d30-59248f940238 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:37.749810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ef1de660-92f9-46ab-8d30-59248f940238 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:37.751373 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1327/2651] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:10:37 2022] DELETE /identity/v3/projects/418128a55411488d879d49bc4c984c71 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:37.755329 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5d4af-beb5-4987-926f-ad2474163d2e service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:37.764700 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5d4af-beb5-4987-926f-ad2474163d2e service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:37.817339 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8baef4-2041-4565-86fa-3a55eabfde76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:37.818215 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8baef4-2041-4565-86fa-3a55eabfde76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:37.818491 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8baef4-2041-4565-86fa-3a55eabfde76 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:37.818758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8baef4-2041-4565-86fa-3a55eabfde76 admin admin] PATH_INFO: `/v3/users/e58afd783c1341f58487c1bdbe98e1f1` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:37.819907 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8baef4-2041-4565-86fa-3a55eabfde76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e58afd783c1341f58487c1bdbe98e1f1)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:37.821799 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8baef4-2041-4565-86fa-3a55eabfde76 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:37.865081 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1e8baef4-2041-4565-86fa-3a55eabfde76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e58afd783c1341f58487c1bdbe98e1f1'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:37.871455 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1e8baef4-2041-4565-86fa-3a55eabfde76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e58afd783c1341f58487c1bdbe98e1f1'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:37.872095 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1e8baef4-2041-4565-86fa-3a55eabfde76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e58afd783c1341f58487c1bdbe98e1f1'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:37.879182 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1e8baef4-2041-4565-86fa-3a55eabfde76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e58afd783c1341f58487c1bdbe98e1f1'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:37.879958 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1e8baef4-2041-4565-86fa-3a55eabfde76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e58afd783c1341f58487c1bdbe98e1f1'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:37.899839 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1325/2652] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:10:37 2022] DELETE /identity/v3/users/e58afd783c1341f58487c1bdbe98e1f1 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:38.211223 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1de660-92f9-46ab-8d30-59248f940238 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:38.212675 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ef1de660-92f9-46ab-8d30-59248f940238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:38.225963 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1de660-92f9-46ab-8d30-59248f940238 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:38.283510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf4823-0cba-4c87-b4d8-fb2b6261c34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:38.284371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf4823-0cba-4c87-b4d8-fb2b6261c34b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:38.284648 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf4823-0cba-4c87-b4d8-fb2b6261c34b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:38.284921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf4823-0cba-4c87-b4d8-fb2b6261c34b admin admin] PATH_INFO: `/v3/projects/28093a9f43fd4573b89980992f5d16d8` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:38.286104 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf4823-0cba-4c87-b4d8-fb2b6261c34b admin admin] RBAC: Authorizing `identity:delete_project(project_id=28093a9f43fd4573b89980992f5d16d8)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:38.288092 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf4823-0cba-4c87-b4d8-fb2b6261c34b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:38.333845 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-65bf4823-0cba-4c87-b4d8-fb2b6261c34b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28093a9f43fd4573b89980992f5d16d8', 'request_id': 'req-65bf4823-0cba-4c87-b4d8-fb2b6261c34b'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:38.338941 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-65bf4823-0cba-4c87-b4d8-fb2b6261c34b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28093a9f43fd4573b89980992f5d16d8', 'request_id': 'req-65bf4823-0cba-4c87-b4d8-fb2b6261c34b'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:38.345526 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-65bf4823-0cba-4c87-b4d8-fb2b6261c34b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28093a9f43fd4573b89980992f5d16d8', 'request_id': 'req-65bf4823-0cba-4c87-b4d8-fb2b6261c34b'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:38.358718 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-65bf4823-0cba-4c87-b4d8-fb2b6261c34b admin admin] The token cache is being invalidate because project 28093a9f43fd4573b89980992f5d16d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:10:38.359310 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-65bf4823-0cba-4c87-b4d8-fb2b6261c34b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:38.359983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-65bf4823-0cba-4c87-b4d8-fb2b6261c34b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:38.361515 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1328/2653] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:10:38 2022] DELETE /identity/v3/projects/28093a9f43fd4573b89980992f5d16d8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:38.366399 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd2682-eb38-4595-b2c0-01e63f590098 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:38.366883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd2682-eb38-4595-b2c0-01e63f590098 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:38.366981 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd2682-eb38-4595-b2c0-01e63f590098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:38.384732 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-2efd2682-eb38-4595-b2c0-01e63f590098 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:38.386564 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2efd2682-eb38-4595-b2c0-01e63f590098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:38.443456 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1326/2654] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:10:38.451947 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffdf17-4549-4a9f-8ea9-0957d018e1e5 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:38.452226 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffdf17-4549-4a9f-8ea9-0957d018e1e5 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:38.452495 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffdf17-4549-4a9f-8ea9-0957d018e1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:38.470840 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-43ffdf17-4549-4a9f-8ea9-0957d018e1e5 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:38.472011 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-43ffdf17-4549-4a9f-8ea9-0957d018e1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:38.479946 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1329/2655] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:10:38.484269 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd2682-eb38-4595-b2c0-01e63f590098 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:38.493708 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd2682-eb38-4595-b2c0-01e63f590098 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:38.498400 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206b881-5309-4bac-bb2c-53f345897a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:38.499044 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206b881-5309-4bac-bb2c-53f345897a7f admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:38.499328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206b881-5309-4bac-bb2c-53f345897a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:38.499604 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206b881-5309-4bac-bb2c-53f345897a7f admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:38.500318 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206b881-5309-4bac-bb2c-53f345897a7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:10:38.500626 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206b881-5309-4bac-bb2c-53f345897a7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:38.502299 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206b881-5309-4bac-bb2c-53f345897a7f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:38.514269 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1327/2656] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:10:38.518536 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffdf17-4549-4a9f-8ea9-0957d018e1e5 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:38.528641 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffdf17-4549-4a9f-8ea9-0957d018e1e5 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:38.533306 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5cb8b7-a1a0-490a-920a-154a72ab7545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:38.537473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cb8b7-a1a0-490a-920a-154a72ab7545 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:38.537473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cb8b7-a1a0-490a-920a-154a72ab7545 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:38.537473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cb8b7-a1a0-490a-920a-154a72ab7545 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:38.537473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5cb8b7-a1a0-490a-920a-154a72ab7545 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:38.537473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5cb8b7-a1a0-490a-920a-154a72ab7545 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:38.563678 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1330/2657] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:10:38 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:10:38.567761 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206b881-5309-4bac-bb2c-53f345897a7f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:38.578066 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206b881-5309-4bac-bb2c-53f345897a7f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:38.633887 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95105ab-b904-4587-b233-8be21278f53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:38.634589 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95105ab-b904-4587-b233-8be21278f53e admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:38.634907 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95105ab-b904-4587-b233-8be21278f53e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:38.635183 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95105ab-b904-4587-b233-8be21278f53e admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:38.636482 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95105ab-b904-4587-b233-8be21278f53e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:38.638414 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95105ab-b904-4587-b233-8be21278f53e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:38.658385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1328/2658] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:10:38 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:10:38.662474 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5cb8b7-a1a0-490a-920a-154a72ab7545 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:38.672423 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5cb8b7-a1a0-490a-920a-154a72ab7545 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:38.676924 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5393cf-00ba-4624-bf03-1c79e6c4e3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:38.677592 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5393cf-00ba-4624-bf03-1c79e6c4e3fa admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:38.677870 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5393cf-00ba-4624-bf03-1c79e6c4e3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:38.678171 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5393cf-00ba-4624-bf03-1c79e6c4e3fa admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:38.679149 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5393cf-00ba-4624-bf03-1c79e6c4e3fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:38.680728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5393cf-00ba-4624-bf03-1c79e6c4e3fa admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:38.691278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1331/2659] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:10:38.695161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95105ab-b904-4587-b233-8be21278f53e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:38.705578 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95105ab-b904-4587-b233-8be21278f53e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:38.710143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61d3a3-cf86-4700-b3fa-3143ca07ea64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:38.710788 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61d3a3-cf86-4700-b3fa-3143ca07ea64 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:38.711067 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61d3a3-cf86-4700-b3fa-3143ca07ea64 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:38.711340 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61d3a3-cf86-4700-b3fa-3143ca07ea64 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:38.712048 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61d3a3-cf86-4700-b3fa-3143ca07ea64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:38.713623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61d3a3-cf86-4700-b3fa-3143ca07ea64 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:38.723586 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1329/2660] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:10:38.727514 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5393cf-00ba-4624-bf03-1c79e6c4e3fa admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:38.737572 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5393cf-00ba-4624-bf03-1c79e6c4e3fa admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:38.742141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515bf73d-f11b-4259-b461-536334f51345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:38.743005 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515bf73d-f11b-4259-b461-536334f51345 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:38.743283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515bf73d-f11b-4259-b461-536334f51345 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:38.743554 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515bf73d-f11b-4259-b461-536334f51345 admin admin] PATH_INFO: `/v3/projects/a1b401834883475e97e6496c9fcf0e96/users/8115e14d002644bdb275d8545d3c45bc/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:38.759262 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515bf73d-f11b-4259-b461-536334f51345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b401834883475e97e6496c9fcf0e96, user_id=8115e14d002644bdb275d8545d3c45bc, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:38.762609 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515bf73d-f11b-4259-b461-536334f51345 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:38.775158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1332/2661] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:10:38 2022] PUT /identity/v3/projects/a1b401834883475e97e6496c9fcf0e96/users/8115e14d002644bdb275d8545d3c45bc/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:38.780094 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22468b25-1c50-47e6-8e79-0550ba4423da None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:38.780381 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22468b25-1c50-47e6-8e79-0550ba4423da None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:38.780654 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22468b25-1c50-47e6-8e79-0550ba4423da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:38.797649 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-22468b25-1c50-47e6-8e79-0550ba4423da None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:38.798857 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-22468b25-1c50-47e6-8e79-0550ba4423da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:38.857810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1330/2662] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:10:38.867346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515bf73d-f11b-4259-b461-536334f51345 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:38.870709 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-515bf73d-f11b-4259-b461-536334f51345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:38.925456 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515bf73d-f11b-4259-b461-536334f51345 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:38.928985 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861821ae-af32-47f4-be86-e716ae4349de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:38.929616 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861821ae-af32-47f4-be86-e716ae4349de service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:38.929895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861821ae-af32-47f4-be86-e716ae4349de service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:38.930196 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861821ae-af32-47f4-be86-e716ae4349de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:38.939353 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861821ae-af32-47f4-be86-e716ae4349de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:38.940760 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861821ae-af32-47f4-be86-e716ae4349de service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:38.950437 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1333/2663] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:10:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:10:39.577323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035c3dd-5e9a-4776-b0ce-41f00cf32ed5 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:39.579165 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035c3dd-5e9a-4776-b0ce-41f00cf32ed5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:39.579455 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035c3dd-5e9a-4776-b0ce-41f00cf32ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:39.609665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-0035c3dd-5e9a-4776-b0ce-41f00cf32ed5 None None] MFA Rules not processed for user `8115e14d002644bdb275d8545d3c45bc`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:39.610915 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0035c3dd-5e9a-4776-b0ce-41f00cf32ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:39.689541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1331/2664] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:10:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:10:39.690984 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861821ae-af32-47f4-be86-e716ae4349de service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:39.701457 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861821ae-af32-47f4-be86-e716ae4349de service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:39.706322 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6327ab7-083b-4b60-80fb-30274ec26efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:39.706980 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6327ab7-083b-4b60-80fb-30274ec26efe admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:39.707260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6327ab7-083b-4b60-80fb-30274ec26efe admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:39.707536 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6327ab7-083b-4b60-80fb-30274ec26efe admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:39.708220 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6327ab7-083b-4b60-80fb-30274ec26efe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:39.710148 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6327ab7-083b-4b60-80fb-30274ec26efe admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:39.740029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1334/2665] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:10:39 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:10:39.744255 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035c3dd-5e9a-4776-b0ce-41f00cf32ed5 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:39.756610 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035c3dd-5e9a-4776-b0ce-41f00cf32ed5 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:39.821306 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322a268-6c5b-40ae-bcd5-d15982cf5209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:39.821968 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322a268-6c5b-40ae-bcd5-d15982cf5209 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:39.822276 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322a268-6c5b-40ae-bcd5-d15982cf5209 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:39.822542 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322a268-6c5b-40ae-bcd5-d15982cf5209 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:39.823800 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322a268-6c5b-40ae-bcd5-d15982cf5209 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:39.825760 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322a268-6c5b-40ae-bcd5-d15982cf5209 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:39.845237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1332/2666] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:10:39 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:10:39.850078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6327ab7-083b-4b60-80fb-30274ec26efe admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:39.865314 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6327ab7-083b-4b60-80fb-30274ec26efe admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:39.866946 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c408e3-e45a-4591-86ab-32d9d93bd456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:39.868517 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c408e3-e45a-4591-86ab-32d9d93bd456 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:39.868796 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c408e3-e45a-4591-86ab-32d9d93bd456 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:39.869067 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c408e3-e45a-4591-86ab-32d9d93bd456 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:39.871541 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c408e3-e45a-4591-86ab-32d9d93bd456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:39.873155 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c408e3-e45a-4591-86ab-32d9d93bd456 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:39.884985 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1335/2667] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:10:39.889753 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322a268-6c5b-40ae-bcd5-d15982cf5209 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:39.899471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322a268-6c5b-40ae-bcd5-d15982cf5209 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:39.904760 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f69762-b92d-489e-9ac9-475b643f8db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:39.905622 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f69762-b92d-489e-9ac9-475b643f8db4 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:39.905895 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f69762-b92d-489e-9ac9-475b643f8db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:39.906196 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f69762-b92d-489e-9ac9-475b643f8db4 admin admin] PATH_INFO: `/v3/projects/7196a3a650b84e75871c817e2f4a7433/users/9fbf9ebc535a4d84ae2fd4825fb99bc7/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:39.920701 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f69762-b92d-489e-9ac9-475b643f8db4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7196a3a650b84e75871c817e2f4a7433, user_id=9fbf9ebc535a4d84ae2fd4825fb99bc7, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:39.923399 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f69762-b92d-489e-9ac9-475b643f8db4 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:39.935453 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1333/2668] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:10:39 2022] PUT /identity/v3/projects/7196a3a650b84e75871c817e2f4a7433/users/9fbf9ebc535a4d84ae2fd4825fb99bc7/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:39.939289 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c408e3-e45a-4591-86ab-32d9d93bd456 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:39.949585 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c408e3-e45a-4591-86ab-32d9d93bd456 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:40.002807 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddfc1d-1670-4a44-bbe3-7175aa4cab6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:40.003454 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddfc1d-1670-4a44-bbe3-7175aa4cab6a admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:40.003734 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddfc1d-1670-4a44-bbe3-7175aa4cab6a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:40.004006 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddfc1d-1670-4a44-bbe3-7175aa4cab6a admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:40.004711 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ddfc1d-1670-4a44-bbe3-7175aa4cab6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:40.006504 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ddfc1d-1670-4a44-bbe3-7175aa4cab6a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:40.016256 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1336/2669] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:10:40.020259 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f69762-b92d-489e-9ac9-475b643f8db4 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:40.030677 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f69762-b92d-489e-9ac9-475b643f8db4 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:40.035122 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39cfae0-0319-441c-9589-d5cb3023c2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:40.035977 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cfae0-0319-441c-9589-d5cb3023c2ad admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:40.036255 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cfae0-0319-441c-9589-d5cb3023c2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:40.036523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cfae0-0319-441c-9589-d5cb3023c2ad admin admin] PATH_INFO: `/v3/projects/7196a3a650b84e75871c817e2f4a7433/users/9fbf9ebc535a4d84ae2fd4825fb99bc7/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:40.038481 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39cfae0-0319-441c-9589-d5cb3023c2ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=7196a3a650b84e75871c817e2f4a7433, user_id=9fbf9ebc535a4d84ae2fd4825fb99bc7, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:40.041072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39cfae0-0319-441c-9589-d5cb3023c2ad admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:40.050688 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1334/2670] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:10:40 2022] PUT /identity/v3/projects/7196a3a650b84e75871c817e2f4a7433/users/9fbf9ebc535a4d84ae2fd4825fb99bc7/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:40.658447 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f968b-2f66-4f3a-a85c-61def738e150 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:40.658786 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f968b-2f66-4f3a-a85c-61def738e150 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:40.659072 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f968b-2f66-4f3a-a85c-61def738e150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:40.685788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5e4f968b-2f66-4f3a-a85c-61def738e150 None None] MFA Rules not processed for user `9fbf9ebc535a4d84ae2fd4825fb99bc7`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:40.687067 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5e4f968b-2f66-4f3a-a85c-61def738e150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:40.753078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1337/2671] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:10:40 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:10:40.764301 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39cfae0-0319-441c-9589-d5cb3023c2ad admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:40.766042 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c39cfae0-0319-441c-9589-d5cb3023c2ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:40.822554 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39cfae0-0319-441c-9589-d5cb3023c2ad admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:40.826098 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee6e07-6a94-4b50-92c1-a89e488a086d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:40.830211 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee6e07-6a94-4b50-92c1-a89e488a086d service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:40.830512 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee6e07-6a94-4b50-92c1-a89e488a086d service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:40.830793 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee6e07-6a94-4b50-92c1-a89e488a086d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:40.889587 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee6e07-6a94-4b50-92c1-a89e488a086d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:40.891355 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee6e07-6a94-4b50-92c1-a89e488a086d service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:40.900982 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1335/2672] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:10:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:10:43.272823 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f968b-2f66-4f3a-a85c-61def738e150 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:43.284089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f968b-2f66-4f3a-a85c-61def738e150 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:43.287638 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dadb81-ae9c-4e1f-927c-afb67444d678 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:43.288268 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dadb81-ae9c-4e1f-927c-afb67444d678 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:43.288544 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dadb81-ae9c-4e1f-927c-afb67444d678 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:43.288812 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dadb81-ae9c-4e1f-927c-afb67444d678 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:43.298298 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dadb81-ae9c-4e1f-927c-afb67444d678 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:43.299719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dadb81-ae9c-4e1f-927c-afb67444d678 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:43.309136 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1338/2673] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:10:43 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:10:43.588514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cda55a-327d-4554-90bd-a4166c576fcd None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:43.589546 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cda55a-327d-4554-90bd-a4166c576fcd None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:43.589726 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cda55a-327d-4554-90bd-a4166c576fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:43.607574 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-25cda55a-327d-4554-90bd-a4166c576fcd None None] MFA Rules not processed for user `1a6f50dcebf146baae8a9f0b5b112795`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:43.608836 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-25cda55a-327d-4554-90bd-a4166c576fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:43.665708 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1336/2674] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 9 23:10:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:10:43.714472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dadb81-ae9c-4e1f-927c-afb67444d678 service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:43.715869 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b1dadb81-ae9c-4e1f-927c-afb67444d678 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:43.786553 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dadb81-ae9c-4e1f-927c-afb67444d678 service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:43.791505 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ade27-0897-4268-8d0c-647a60f40f75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cb6c294ce1842b68a6bc73ac381ed6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:43.792146 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ade27-0897-4268-8d0c-647a60f40f75 service swift] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:43.792427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ade27-0897-4268-8d0c-647a60f40f75 service swift] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:43.792700 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ade27-0897-4268-8d0c-647a60f40f75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:43.800479 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ade27-0897-4268-8d0c-647a60f40f75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:43.802034 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ade27-0897-4268-8d0c-647a60f40f75 service swift] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:43.809735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1339/2675] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 9 23:10:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 23:10:56.694874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cda55a-327d-4554-90bd-a4166c576fcd None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:56.768294 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cda55a-327d-4554-90bd-a4166c576fcd None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:56.775583 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050790c-ed26-4e38-a1e3-7e7aa4eb9dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:56.778306 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050790c-ed26-4e38-a1e3-7e7aa4eb9dea admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:56.779182 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050790c-ed26-4e38-a1e3-7e7aa4eb9dea admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:56.779839 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050790c-ed26-4e38-a1e3-7e7aa4eb9dea admin admin] PATH_INFO: `/v3/users/8115e14d002644bdb275d8545d3c45bc` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:56.781991 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050790c-ed26-4e38-a1e3-7e7aa4eb9dea admin admin] RBAC: Authorizing `identity:delete_user(user_id=8115e14d002644bdb275d8545d3c45bc)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:56.784517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050790c-ed26-4e38-a1e3-7e7aa4eb9dea admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:56.845650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-0050790c-ed26-4e38-a1e3-7e7aa4eb9dea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8115e14d002644bdb275d8545d3c45bc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:56.854739 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-0050790c-ed26-4e38-a1e3-7e7aa4eb9dea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8115e14d002644bdb275d8545d3c45bc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:56.855928 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-0050790c-ed26-4e38-a1e3-7e7aa4eb9dea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8115e14d002644bdb275d8545d3c45bc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:56.864591 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-0050790c-ed26-4e38-a1e3-7e7aa4eb9dea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8115e14d002644bdb275d8545d3c45bc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:56.865468 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-0050790c-ed26-4e38-a1e3-7e7aa4eb9dea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8115e14d002644bdb275d8545d3c45bc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:56.887651 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1337/2676] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:10:56 2022] DELETE /identity/v3/users/8115e14d002644bdb275d8545d3c45bc => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:57.078629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ade27-0897-4268-8d0c-647a60f40f75 service swift] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:57.092519 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ade27-0897-4268-8d0c-647a60f40f75 service swift] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:57.153567 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88421192-1848-4a26-a4c3-e852e043280e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:57.154471 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88421192-1848-4a26-a4c3-e852e043280e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:57.154752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88421192-1848-4a26-a4c3-e852e043280e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:57.155019 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88421192-1848-4a26-a4c3-e852e043280e admin admin] PATH_INFO: `/v3/projects/a1b401834883475e97e6496c9fcf0e96` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:57.156243 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88421192-1848-4a26-a4c3-e852e043280e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1b401834883475e97e6496c9fcf0e96)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:57.158213 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88421192-1848-4a26-a4c3-e852e043280e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:57.212350 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-88421192-1848-4a26-a4c3-e852e043280e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1b401834883475e97e6496c9fcf0e96', 'request_id': 'req-88421192-1848-4a26-a4c3-e852e043280e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:57.217951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-88421192-1848-4a26-a4c3-e852e043280e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1b401834883475e97e6496c9fcf0e96', 'request_id': 'req-88421192-1848-4a26-a4c3-e852e043280e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:57.224468 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-88421192-1848-4a26-a4c3-e852e043280e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1b401834883475e97e6496c9fcf0e96', 'request_id': 'req-88421192-1848-4a26-a4c3-e852e043280e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:57.238939 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-88421192-1848-4a26-a4c3-e852e043280e admin admin] The token cache is being invalidate because project a1b401834883475e97e6496c9fcf0e96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:10:57.239290 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-88421192-1848-4a26-a4c3-e852e043280e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:57.239978 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-88421192-1848-4a26-a4c3-e852e043280e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:57.242153 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1340/2677] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:10:57 2022] DELETE /identity/v3/projects/a1b401834883475e97e6496c9fcf0e96 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:57.246630 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050790c-ed26-4e38-a1e3-7e7aa4eb9dea admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:57.248079 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0050790c-ed26-4e38-a1e3-7e7aa4eb9dea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:57.261480 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050790c-ed26-4e38-a1e3-7e7aa4eb9dea admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:57.334898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f691254-126a-4147-884e-ef9aaa7b0237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:57.335785 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f691254-126a-4147-884e-ef9aaa7b0237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:57.336060 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f691254-126a-4147-884e-ef9aaa7b0237 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:57.336328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f691254-126a-4147-884e-ef9aaa7b0237 admin admin] PATH_INFO: `/v3/users/9fbf9ebc535a4d84ae2fd4825fb99bc7` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:57.337496 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f691254-126a-4147-884e-ef9aaa7b0237 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fbf9ebc535a4d84ae2fd4825fb99bc7)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:57.340043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f691254-126a-4147-884e-ef9aaa7b0237 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:57.393949 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4f691254-126a-4147-884e-ef9aaa7b0237 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fbf9ebc535a4d84ae2fd4825fb99bc7'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:57.404481 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4f691254-126a-4147-884e-ef9aaa7b0237 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fbf9ebc535a4d84ae2fd4825fb99bc7'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:57.406195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4f691254-126a-4147-884e-ef9aaa7b0237 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fbf9ebc535a4d84ae2fd4825fb99bc7'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:57.415183 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4f691254-126a-4147-884e-ef9aaa7b0237 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fbf9ebc535a4d84ae2fd4825fb99bc7'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:57.416022 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4f691254-126a-4147-884e-ef9aaa7b0237 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fbf9ebc535a4d84ae2fd4825fb99bc7'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:57.436058 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1338/2678] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:10:57 2022] DELETE /identity/v3/users/9fbf9ebc535a4d84ae2fd4825fb99bc7 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:57.638146 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88421192-1848-4a26-a4c3-e852e043280e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:57.647384 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-88421192-1848-4a26-a4c3-e852e043280e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:57.683670 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88421192-1848-4a26-a4c3-e852e043280e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:57.743103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c27eb-ce87-4fbd-b04d-fa558c82dffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:57.744111 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c27eb-ce87-4fbd-b04d-fa558c82dffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:57.744543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c27eb-ce87-4fbd-b04d-fa558c82dffc admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:57.744811 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c27eb-ce87-4fbd-b04d-fa558c82dffc admin admin] PATH_INFO: `/v3/projects/7196a3a650b84e75871c817e2f4a7433` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:57.746069 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c27eb-ce87-4fbd-b04d-fa558c82dffc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7196a3a650b84e75871c817e2f4a7433)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:57.748200 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c27eb-ce87-4fbd-b04d-fa558c82dffc admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:57.808103 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-999c27eb-ce87-4fbd-b04d-fa558c82dffc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7196a3a650b84e75871c817e2f4a7433', 'request_id': 'req-999c27eb-ce87-4fbd-b04d-fa558c82dffc'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:57.813772 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-999c27eb-ce87-4fbd-b04d-fa558c82dffc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7196a3a650b84e75871c817e2f4a7433', 'request_id': 'req-999c27eb-ce87-4fbd-b04d-fa558c82dffc'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:57.820605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-999c27eb-ce87-4fbd-b04d-fa558c82dffc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7196a3a650b84e75871c817e2f4a7433', 'request_id': 'req-999c27eb-ce87-4fbd-b04d-fa558c82dffc'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:57.836998 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-999c27eb-ce87-4fbd-b04d-fa558c82dffc admin admin] The token cache is being invalidate because project 7196a3a650b84e75871c817e2f4a7433 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:10:57.837273 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-999c27eb-ce87-4fbd-b04d-fa558c82dffc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:57.837916 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-999c27eb-ce87-4fbd-b04d-fa558c82dffc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:10:57.840440 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1341/2679] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:10:57 2022] DELETE /identity/v3/projects/7196a3a650b84e75871c817e2f4a7433 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:57.845131 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806c435-9373-4694-b442-859fb59bd459 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:57.845646 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806c435-9373-4694-b442-859fb59bd459 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:57.845748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806c435-9373-4694-b442-859fb59bd459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:57.868644 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a806c435-9373-4694-b442-859fb59bd459 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:57.870153 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a806c435-9373-4694-b442-859fb59bd459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:57.932188 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1339/2680] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:10:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:10:57.940395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e8df7-05d5-4814-934e-023ffdf20714 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:57.940777 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e8df7-05d5-4814-934e-023ffdf20714 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:57.941154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e8df7-05d5-4814-934e-023ffdf20714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:57.962815 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-806e8df7-05d5-4814-934e-023ffdf20714 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:57.964780 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-806e8df7-05d5-4814-934e-023ffdf20714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:57.976990 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1342/2681] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:10:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:10:57.982191 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806c435-9373-4694-b442-859fb59bd459 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:57.992522 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806c435-9373-4694-b442-859fb59bd459 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:57.998557 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe596f69-44da-4b04-8613-45cb9e52ae40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:57.999522 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe596f69-44da-4b04-8613-45cb9e52ae40 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:57.999803 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe596f69-44da-4b04-8613-45cb9e52ae40 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:58.000076 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe596f69-44da-4b04-8613-45cb9e52ae40 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:58.000820 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe596f69-44da-4b04-8613-45cb9e52ae40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:10:58.001423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe596f69-44da-4b04-8613-45cb9e52ae40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:58.002967 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe596f69-44da-4b04-8613-45cb9e52ae40 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:58.019227 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1340/2682] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:10:58.026442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e8df7-05d5-4814-934e-023ffdf20714 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:58.035740 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e8df7-05d5-4814-934e-023ffdf20714 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:58.040959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bdedc-5402-4f4e-8fc1-71ef3c92105d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:58.041618 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bdedc-5402-4f4e-8fc1-71ef3c92105d admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:58.041995 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bdedc-5402-4f4e-8fc1-71ef3c92105d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:58.042202 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bdedc-5402-4f4e-8fc1-71ef3c92105d admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:58.042908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3bdedc-5402-4f4e-8fc1-71ef3c92105d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:58.045514 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3bdedc-5402-4f4e-8fc1-71ef3c92105d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:58.071911 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1343/2683] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:10:58 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:10:58.076637 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe596f69-44da-4b04-8613-45cb9e52ae40 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:58.086873 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe596f69-44da-4b04-8613-45cb9e52ae40 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:58.149232 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1771eb8-25f0-4477-be2c-0d19607bcff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:58.149899 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1771eb8-25f0-4477-be2c-0d19607bcff3 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:58.150237 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1771eb8-25f0-4477-be2c-0d19607bcff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:58.151052 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1771eb8-25f0-4477-be2c-0d19607bcff3 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:58.151191 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1771eb8-25f0-4477-be2c-0d19607bcff3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:58.153083 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1771eb8-25f0-4477-be2c-0d19607bcff3 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:58.174895 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1341/2684] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:10:58 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:10:58.179012 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bdedc-5402-4f4e-8fc1-71ef3c92105d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:58.190004 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bdedc-5402-4f4e-8fc1-71ef3c92105d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:58.195453 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207392da-478e-436c-8e23-f56707f6ab8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:58.196422 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207392da-478e-436c-8e23-f56707f6ab8e admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:58.196705 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207392da-478e-436c-8e23-f56707f6ab8e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:58.196980 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207392da-478e-436c-8e23-f56707f6ab8e admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:58.198386 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207392da-478e-436c-8e23-f56707f6ab8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:58.199973 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207392da-478e-436c-8e23-f56707f6ab8e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:58.211372 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1344/2685] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:10:58.214985 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1771eb8-25f0-4477-be2c-0d19607bcff3 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:58.224947 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1771eb8-25f0-4477-be2c-0d19607bcff3 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:58.229609 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994c0e5-d9d1-4ebf-b2db-40fa0782b75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:58.230293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994c0e5-d9d1-4ebf-b2db-40fa0782b75d admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:58.230849 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994c0e5-d9d1-4ebf-b2db-40fa0782b75d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:58.231124 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994c0e5-d9d1-4ebf-b2db-40fa0782b75d admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:58.231875 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994c0e5-d9d1-4ebf-b2db-40fa0782b75d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:58.233534 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994c0e5-d9d1-4ebf-b2db-40fa0782b75d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:58.245257 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1342/2686] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:10:58.248279 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207392da-478e-436c-8e23-f56707f6ab8e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:58.258342 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207392da-478e-436c-8e23-f56707f6ab8e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:58.263949 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63cbdf-32b5-4f0b-b425-167f99540252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:58.264840 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63cbdf-32b5-4f0b-b425-167f99540252 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:58.265122 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63cbdf-32b5-4f0b-b425-167f99540252 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:58.265399 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63cbdf-32b5-4f0b-b425-167f99540252 admin admin] PATH_INFO: `/v3/projects/8a336aee15864852bcc077d916ea298f/users/5217118a9fbb4a76a8039b941d76acca/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:58.280083 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec63cbdf-32b5-4f0b-b425-167f99540252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a336aee15864852bcc077d916ea298f, user_id=5217118a9fbb4a76a8039b941d76acca, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:58.282788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec63cbdf-32b5-4f0b-b425-167f99540252 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:58.304853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1345/2687] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:10:58 2022] PUT /identity/v3/projects/8a336aee15864852bcc077d916ea298f/users/5217118a9fbb4a76a8039b941d76acca/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:10:58.313634 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072a0b5-b255-45cc-82cb-fe1100f9fbca None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:58.313948 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072a0b5-b255-45cc-82cb-fe1100f9fbca None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:58.314380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072a0b5-b255-45cc-82cb-fe1100f9fbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:58.344692 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-9072a0b5-b255-45cc-82cb-fe1100f9fbca None None] MFA Rules not processed for user `5217118a9fbb4a76a8039b941d76acca`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:58.346011 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9072a0b5-b255-45cc-82cb-fe1100f9fbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:58.416887 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1343/2688] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:10:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:10:58.427868 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63cbdf-32b5-4f0b-b425-167f99540252 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:58.429601 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ec63cbdf-32b5-4f0b-b425-167f99540252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:58.492490 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63cbdf-32b5-4f0b-b425-167f99540252 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:58.496966 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e6b28-ce45-4906-8e2d-068d90558c0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:58.497621 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e6b28-ce45-4906-8e2d-068d90558c0f service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:58.497895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e6b28-ce45-4906-8e2d-068d90558c0f service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:58.498228 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e6b28-ce45-4906-8e2d-068d90558c0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:58.506861 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e6b28-ce45-4906-8e2d-068d90558c0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:58.508286 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e6b28-ce45-4906-8e2d-068d90558c0f service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:58.517359 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1346/2689] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:10:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:10:59.471835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe29043-d332-4b39-8123-31aa41c9fa6b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:59.472298 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe29043-d332-4b39-8123-31aa41c9fa6b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:59.472401 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe29043-d332-4b39-8123-31aa41c9fa6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:59.490059 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-8fe29043-d332-4b39-8123-31aa41c9fa6b None None] MFA Rules not processed for user `1a6f50dcebf146baae8a9f0b5b112795`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:10:59.491296 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8fe29043-d332-4b39-8123-31aa41c9fa6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:59.544385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1344/2690] 10.222.0.23 () {58 vars in 967 bytes} [Mon May 9 23:10:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:10:59.556793 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e6b28-ce45-4906-8e2d-068d90558c0f service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:10:59.558238 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-101e6b28-ce45-4906-8e2d-068d90558c0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:10:59.631823 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e6b28-ce45-4906-8e2d-068d90558c0f service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:10:59.637623 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6770816-90d1-49f8-8f41-67818c9b62c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cb6c294ce1842b68a6bc73ac381ed6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:10:59.638308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6770816-90d1-49f8-8f41-67818c9b62c5 service swift] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:10:59.638592 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6770816-90d1-49f8-8f41-67818c9b62c5 service swift] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:10:59.639575 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6770816-90d1-49f8-8f41-67818c9b62c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:10:59.647373 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6770816-90d1-49f8-8f41-67818c9b62c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:10:59.648954 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6770816-90d1-49f8-8f41-67818c9b62c5 service swift] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:10:59.657249 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1347/2691] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 9 23:10:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 23:11:01.479278 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe29043-d332-4b39-8123-31aa41c9fa6b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:01.571595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe29043-d332-4b39-8123-31aa41c9fa6b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:01.576963 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ed863-d0c2-4ba0-b168-2ef022f87727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:01.577982 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ed863-d0c2-4ba0-b168-2ef022f87727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:01.578370 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ed863-d0c2-4ba0-b168-2ef022f87727 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:01.578695 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ed863-d0c2-4ba0-b168-2ef022f87727 admin admin] PATH_INFO: `/v3/users/5217118a9fbb4a76a8039b941d76acca` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:01.580463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ed863-d0c2-4ba0-b168-2ef022f87727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5217118a9fbb4a76a8039b941d76acca)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:01.595973 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ed863-d0c2-4ba0-b168-2ef022f87727 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:01.685486 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-542ed863-d0c2-4ba0-b168-2ef022f87727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5217118a9fbb4a76a8039b941d76acca'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:01.696422 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-542ed863-d0c2-4ba0-b168-2ef022f87727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5217118a9fbb4a76a8039b941d76acca'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:01.698003 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-542ed863-d0c2-4ba0-b168-2ef022f87727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5217118a9fbb4a76a8039b941d76acca'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:01.707293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-542ed863-d0c2-4ba0-b168-2ef022f87727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5217118a9fbb4a76a8039b941d76acca'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:01.707991 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-542ed863-d0c2-4ba0-b168-2ef022f87727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5217118a9fbb4a76a8039b941d76acca'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:01.735768 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1345/2692] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:11:01 2022] DELETE /identity/v3/users/5217118a9fbb4a76a8039b941d76acca => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:11:01.740055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a833148-df72-44f7-baef-9823a8c45e17 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:01.740528 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a833148-df72-44f7-baef-9823a8c45e17 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:01.740657 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a833148-df72-44f7-baef-9823a8c45e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:01.758200 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3a833148-df72-44f7-baef-9823a8c45e17 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:11:01.759560 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3a833148-df72-44f7-baef-9823a8c45e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:11:01.826862 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1348/2693] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:11:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:11:01.837242 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ed863-d0c2-4ba0-b168-2ef022f87727 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:01.839349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-542ed863-d0c2-4ba0-b168-2ef022f87727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:11:01.854863 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a833148-df72-44f7-baef-9823a8c45e17 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:01.856852 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3a833148-df72-44f7-baef-9823a8c45e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:11:01.880099 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a833148-df72-44f7-baef-9823a8c45e17 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:01.887340 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167ef21-04c5-4576-8d47-ef238e83efde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:01.888412 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167ef21-04c5-4576-8d47-ef238e83efde admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:01.888886 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167ef21-04c5-4576-8d47-ef238e83efde admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:01.889232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167ef21-04c5-4576-8d47-ef238e83efde admin admin] PATH_INFO: `/v3/users/8f0d8c487e3a4570abea4ad26bcaa34a` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:01.890589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167ef21-04c5-4576-8d47-ef238e83efde admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f0d8c487e3a4570abea4ad26bcaa34a)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:01.892756 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167ef21-04c5-4576-8d47-ef238e83efde admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:01.905554 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ed863-d0c2-4ba0-b168-2ef022f87727 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:01.909892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66358a5f-bee0-456c-8311-0ea2aa3676e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:01.910567 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66358a5f-bee0-456c-8311-0ea2aa3676e4 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:01.910844 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66358a5f-bee0-456c-8311-0ea2aa3676e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:01.911108 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66358a5f-bee0-456c-8311-0ea2aa3676e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:01.920772 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66358a5f-bee0-456c-8311-0ea2aa3676e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:01.922610 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66358a5f-bee0-456c-8311-0ea2aa3676e4 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:01.936523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1346/2694] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:11:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:11:01.955680 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e167ef21-04c5-4576-8d47-ef238e83efde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f0d8c487e3a4570abea4ad26bcaa34a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:01.963934 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e167ef21-04c5-4576-8d47-ef238e83efde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f0d8c487e3a4570abea4ad26bcaa34a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:01.974969 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e167ef21-04c5-4576-8d47-ef238e83efde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f0d8c487e3a4570abea4ad26bcaa34a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:01.976014 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e167ef21-04c5-4576-8d47-ef238e83efde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f0d8c487e3a4570abea4ad26bcaa34a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:01.992076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e167ef21-04c5-4576-8d47-ef238e83efde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f0d8c487e3a4570abea4ad26bcaa34a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:02.004489 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1349/2695] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:11:01 2022] DELETE /identity/v3/users/8f0d8c487e3a4570abea4ad26bcaa34a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:11:02.244918 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66358a5f-bee0-456c-8311-0ea2aa3676e4 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:02.256316 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66358a5f-bee0-456c-8311-0ea2aa3676e4 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:02.334645 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc22be-98d0-441a-be21-595428df135a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:02.335674 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc22be-98d0-441a-be21-595428df135a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:02.335962 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc22be-98d0-441a-be21-595428df135a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:02.339760 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc22be-98d0-441a-be21-595428df135a admin admin] PATH_INFO: `/v3/projects/f99e7d13264c4801bbd78669d85fd6ce` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:02.341001 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167ef21-04c5-4576-8d47-ef238e83efde admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:02.341726 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc22be-98d0-441a-be21-595428df135a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f99e7d13264c4801bbd78669d85fd6ce)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:02.342691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e167ef21-04c5-4576-8d47-ef238e83efde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:11:02.343728 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc22be-98d0-441a-be21-595428df135a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:02.363787 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167ef21-04c5-4576-8d47-ef238e83efde admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:02.369136 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a721dbbd-3dd7-4e09-985c-af4ab8359a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:02.371055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721dbbd-3dd7-4e09-985c-af4ab8359a91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:02.371706 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721dbbd-3dd7-4e09-985c-af4ab8359a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:02.372226 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721dbbd-3dd7-4e09-985c-af4ab8359a91 admin admin] PATH_INFO: `/v3/projects/8a336aee15864852bcc077d916ea298f` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:02.373635 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a721dbbd-3dd7-4e09-985c-af4ab8359a91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a336aee15864852bcc077d916ea298f)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:02.376006 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a721dbbd-3dd7-4e09-985c-af4ab8359a91 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:02.407113 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e2cc22be-98d0-441a-be21-595428df135a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f99e7d13264c4801bbd78669d85fd6ce', 'request_id': 'req-e2cc22be-98d0-441a-be21-595428df135a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:02.413870 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e2cc22be-98d0-441a-be21-595428df135a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f99e7d13264c4801bbd78669d85fd6ce', 'request_id': 'req-e2cc22be-98d0-441a-be21-595428df135a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:02.420045 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e2cc22be-98d0-441a-be21-595428df135a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f99e7d13264c4801bbd78669d85fd6ce', 'request_id': 'req-e2cc22be-98d0-441a-be21-595428df135a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:02.430588 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a721dbbd-3dd7-4e09-985c-af4ab8359a91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a336aee15864852bcc077d916ea298f', 'request_id': 'req-a721dbbd-3dd7-4e09-985c-af4ab8359a91'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:02.432719 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e2cc22be-98d0-441a-be21-595428df135a admin admin] The token cache is being invalidate because project f99e7d13264c4801bbd78669d85fd6ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:11:02.433043 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e2cc22be-98d0-441a-be21-595428df135a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:02.433745 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e2cc22be-98d0-441a-be21-595428df135a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:02.435467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1347/2696] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:11:02 2022] DELETE /identity/v3/projects/f99e7d13264c4801bbd78669d85fd6ce => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:11:02.436691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a721dbbd-3dd7-4e09-985c-af4ab8359a91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a336aee15864852bcc077d916ea298f', 'request_id': 'req-a721dbbd-3dd7-4e09-985c-af4ab8359a91'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:02.440194 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ed962-7fc3-4f6c-8b24-102eee3d3d7e None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:02.440473 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ed962-7fc3-4f6c-8b24-102eee3d3d7e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:02.440751 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ed962-7fc3-4f6c-8b24-102eee3d3d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:02.443431 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a721dbbd-3dd7-4e09-985c-af4ab8359a91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a336aee15864852bcc077d916ea298f', 'request_id': 'req-a721dbbd-3dd7-4e09-985c-af4ab8359a91'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:02.458815 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a721dbbd-3dd7-4e09-985c-af4ab8359a91 admin admin] The token cache is being invalidate because project 8a336aee15864852bcc077d916ea298f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:11:02.459107 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a721dbbd-3dd7-4e09-985c-af4ab8359a91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:02.459736 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a721dbbd-3dd7-4e09-985c-af4ab8359a91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:02.461502 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1350/2697] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:11:02 2022] DELETE /identity/v3/projects/8a336aee15864852bcc077d916ea298f => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:11:02.467791 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32526c7-7020-4ee9-b74e-fa3ca16a10fc None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:02.468082 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32526c7-7020-4ee9-b74e-fa3ca16a10fc None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:02.468382 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32526c7-7020-4ee9-b74e-fa3ca16a10fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:02.470930 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-5d7ed962-7fc3-4f6c-8b24-102eee3d3d7e None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:11:02.472150 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5d7ed962-7fc3-4f6c-8b24-102eee3d3d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:11:02.487264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-d32526c7-7020-4ee9-b74e-fa3ca16a10fc None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:11:02.488619 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d32526c7-7020-4ee9-b74e-fa3ca16a10fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:11:02.549284 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1348/2698] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:11:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:11:02.565309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1351/2699] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:11:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:11:02.574327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f8733-9500-4c1c-be21-82bd3aff1635 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:02.574661 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f8733-9500-4c1c-be21-82bd3aff1635 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:02.574891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f8733-9500-4c1c-be21-82bd3aff1635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:02.597064 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-f80f8733-9500-4c1c-be21-82bd3aff1635 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:11:02.598426 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f80f8733-9500-4c1c-be21-82bd3aff1635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:11:02.607648 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1349/2700] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:11:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:11:02.611835 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f8733-9500-4c1c-be21-82bd3aff1635 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:02.611963 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675083a-f7a6-4b02-9e8f-c9e7f54c83d7 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:02.612205 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675083a-f7a6-4b02-9e8f-c9e7f54c83d7 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:02.612484 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675083a-f7a6-4b02-9e8f-c9e7f54c83d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:02.622778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f8733-9500-4c1c-be21-82bd3aff1635 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:02.631251 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83106a29-b305-4eae-a75f-5372247f475e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:02.631914 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83106a29-b305-4eae-a75f-5372247f475e admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:02.632195 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83106a29-b305-4eae-a75f-5372247f475e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:02.632466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83106a29-b305-4eae-a75f-5372247f475e admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:02.633185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83106a29-b305-4eae-a75f-5372247f475e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:11:02.633492 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83106a29-b305-4eae-a75f-5372247f475e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:02.635821 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83106a29-b305-4eae-a75f-5372247f475e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:02.636809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-9675083a-f7a6-4b02-9e8f-c9e7f54c83d7 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:11:02.638276 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9675083a-f7a6-4b02-9e8f-c9e7f54c83d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:11:02.646428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1352/2701] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:11:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:11:02.648604 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1350/2702] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:11:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:11:02.650621 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675083a-f7a6-4b02-9e8f-c9e7f54c83d7 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:02.652705 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83106a29-b305-4eae-a75f-5372247f475e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:02.659790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675083a-f7a6-4b02-9e8f-c9e7f54c83d7 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:02.663024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83106a29-b305-4eae-a75f-5372247f475e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:02.664677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5157b-dbf2-424b-81fc-7e2d0a749a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:02.665456 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5157b-dbf2-424b-81fc-7e2d0a749a84 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:02.665813 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5157b-dbf2-424b-81fc-7e2d0a749a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:02.666220 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5157b-dbf2-424b-81fc-7e2d0a749a84 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:02.667013 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f5157b-dbf2-424b-81fc-7e2d0a749a84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:11:02.667387 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f5157b-dbf2-424b-81fc-7e2d0a749a84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:02.669114 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f5157b-dbf2-424b-81fc-7e2d0a749a84 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:02.669465 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cbb48-db3e-4d8c-a960-04d1d363e08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:02.670951 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cbb48-db3e-4d8c-a960-04d1d363e08a admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:02.671439 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cbb48-db3e-4d8c-a960-04d1d363e08a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:02.671883 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cbb48-db3e-4d8c-a960-04d1d363e08a admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:02.672581 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9cbb48-db3e-4d8c-a960-04d1d363e08a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:02.674628 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9cbb48-db3e-4d8c-a960-04d1d363e08a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:02.685643 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1353/2703] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:11:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 23:11:02.686829 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5157b-dbf2-424b-81fc-7e2d0a749a84 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:02.696805 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5157b-dbf2-424b-81fc-7e2d0a749a84 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:02.701789 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1351/2704] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:11:02 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:11:02.705164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f330cb7-be24-481d-996b-fbdcdb38db2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:02.705814 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f330cb7-be24-481d-996b-fbdcdb38db2e admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:02.706144 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cbb48-db3e-4d8c-a960-04d1d363e08a admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:02.706298 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f330cb7-be24-481d-996b-fbdcdb38db2e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:02.706521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f330cb7-be24-481d-996b-fbdcdb38db2e admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:02.707207 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f330cb7-be24-481d-996b-fbdcdb38db2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:02.709106 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f330cb7-be24-481d-996b-fbdcdb38db2e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:02.739301 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1354/2705] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:11:02 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:11:02.744090 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f330cb7-be24-481d-996b-fbdcdb38db2e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:02.774055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cbb48-db3e-4d8c-a960-04d1d363e08a admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:02.806375 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f330cb7-be24-481d-996b-fbdcdb38db2e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:02.811494 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf659a-8432-457d-8669-d8b74e7b2e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:02.812439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf659a-8432-457d-8669-d8b74e7b2e52 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:02.812439 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf659a-8432-457d-8669-d8b74e7b2e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:02.812682 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf659a-8432-457d-8669-d8b74e7b2e52 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:02.813346 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf659a-8432-457d-8669-d8b74e7b2e52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:02.815295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf659a-8432-457d-8669-d8b74e7b2e52 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:02.829451 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03273065-e310-4fd6-ac65-5ba14fc7f88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:02.830238 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03273065-e310-4fd6-ac65-5ba14fc7f88f admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:02.830547 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03273065-e310-4fd6-ac65-5ba14fc7f88f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:02.830893 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03273065-e310-4fd6-ac65-5ba14fc7f88f admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:02.831604 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03273065-e310-4fd6-ac65-5ba14fc7f88f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:02.833718 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03273065-e310-4fd6-ac65-5ba14fc7f88f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:02.838787 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1355/2706] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:11:02 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:11:02.839937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf659a-8432-457d-8669-d8b74e7b2e52 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:02.851799 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf659a-8432-457d-8669-d8b74e7b2e52 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:02.854010 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1352/2707] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:11:02 2022] POST /identity/v3/users => generated 296 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:11:02.857646 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1cff68-fb6d-42c6-bbd3-f70b88fef07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:02.858168 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03273065-e310-4fd6-ac65-5ba14fc7f88f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:02.858367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cff68-fb6d-42c6-bbd3-f70b88fef07d admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:02.858629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cff68-fb6d-42c6-bbd3-f70b88fef07d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:02.858907 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cff68-fb6d-42c6-bbd3-f70b88fef07d admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:02.859624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1cff68-fb6d-42c6-bbd3-f70b88fef07d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:02.861199 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1cff68-fb6d-42c6-bbd3-f70b88fef07d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:02.867500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03273065-e310-4fd6-ac65-5ba14fc7f88f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:02.872043 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1356/2708] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:11:02.875039 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b60e3-fd9d-417d-91bd-70f056231323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:02.875730 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b60e3-fd9d-417d-91bd-70f056231323 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:02.876019 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b60e3-fd9d-417d-91bd-70f056231323 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:02.876144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1cff68-fb6d-42c6-bbd3-f70b88fef07d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:02.876296 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b60e3-fd9d-417d-91bd-70f056231323 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:02.877013 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b60e3-fd9d-417d-91bd-70f056231323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:02.878627 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b60e3-fd9d-417d-91bd-70f056231323 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:02.886367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1cff68-fb6d-42c6-bbd3-f70b88fef07d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:02.889926 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1353/2709] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:11:02.891751 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421bbd4c-47a7-4850-8bbe-60e683b2d4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:02.892400 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bbd4c-47a7-4850-8bbe-60e683b2d4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:02.892677 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bbd4c-47a7-4850-8bbe-60e683b2d4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:02.892948 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bbd4c-47a7-4850-8bbe-60e683b2d4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:02.893658 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421bbd4c-47a7-4850-8bbe-60e683b2d4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:02.893786 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b60e3-fd9d-417d-91bd-70f056231323 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:02.895258 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421bbd4c-47a7-4850-8bbe-60e683b2d4d3 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:02.903139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b60e3-fd9d-417d-91bd-70f056231323 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:02.905890 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1357/2710] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:11:02.908357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8487bd-482d-4cb6-b5cd-ac3414ffb285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:02.909079 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8487bd-482d-4cb6-b5cd-ac3414ffb285 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:02.909426 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8487bd-482d-4cb6-b5cd-ac3414ffb285 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:02.909766 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8487bd-482d-4cb6-b5cd-ac3414ffb285 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:02.909884 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421bbd4c-47a7-4850-8bbe-60e683b2d4d3 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:02.910643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8487bd-482d-4cb6-b5cd-ac3414ffb285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:02.912273 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8487bd-482d-4cb6-b5cd-ac3414ffb285 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:02.919347 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421bbd4c-47a7-4850-8bbe-60e683b2d4d3 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:02.923749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1354/2711] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:11:02.924639 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13f9d1-b81d-4435-951a-41d142679b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:02.925507 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13f9d1-b81d-4435-951a-41d142679b7b admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:02.925792 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13f9d1-b81d-4435-951a-41d142679b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:02.926145 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13f9d1-b81d-4435-951a-41d142679b7b admin admin] PATH_INFO: `/v3/projects/130ce8965a764f6d8910c2bba4561534/users/4e2656d339a74bd882c2226251ac3d36/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:02.927893 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8487bd-482d-4cb6-b5cd-ac3414ffb285 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:02.937893 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8487bd-482d-4cb6-b5cd-ac3414ffb285 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:02.941485 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13f9d1-b81d-4435-951a-41d142679b7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=130ce8965a764f6d8910c2bba4561534, user_id=4e2656d339a74bd882c2226251ac3d36, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:02.942688 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf03ef75-28f5-48f3-8191-65856da8eb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:02.943550 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf03ef75-28f5-48f3-8191-65856da8eb29 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:02.943824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf03ef75-28f5-48f3-8191-65856da8eb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:02.944089 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf03ef75-28f5-48f3-8191-65856da8eb29 admin admin] PATH_INFO: `/v3/projects/f0a7b934e5744e34b881025ab35a6799/users/fbc2d50f87044733ba33c2a07c3ede20/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:02.944199 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13f9d1-b81d-4435-951a-41d142679b7b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:02.957747 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1358/2712] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:11:02 2022] PUT /identity/v3/projects/130ce8965a764f6d8910c2bba4561534/users/4e2656d339a74bd882c2226251ac3d36/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:11:02.959504 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf03ef75-28f5-48f3-8191-65856da8eb29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0a7b934e5744e34b881025ab35a6799, user_id=fbc2d50f87044733ba33c2a07c3ede20, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:02.962273 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf03ef75-28f5-48f3-8191-65856da8eb29 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:02.962982 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48e3c3-f192-47fc-a0bb-dc2b93becaee None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:02.963676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48e3c3-f192-47fc-a0bb-dc2b93becaee None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:02.963969 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48e3c3-f192-47fc-a0bb-dc2b93becaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:02.975673 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1355/2713] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:11:02 2022] PUT /identity/v3/projects/f0a7b934e5744e34b881025ab35a6799/users/fbc2d50f87044733ba33c2a07c3ede20/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:11:02.982201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-1d48e3c3-f192-47fc-a0bb-dc2b93becaee None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:11:02.983417 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1d48e3c3-f192-47fc-a0bb-dc2b93becaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:11:03.040524 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e0c2a-401f-46de-9d45-7017368061b1 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:03.040810 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e0c2a-401f-46de-9d45-7017368061b1 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:03.041616 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e0c2a-401f-46de-9d45-7017368061b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:03.048373 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1359/2714] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:11:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:11:03.063127 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48e3c3-f192-47fc-a0bb-dc2b93becaee None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:03.064571 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1d48e3c3-f192-47fc-a0bb-dc2b93becaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:11:03.076352 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-9d2e0c2a-401f-46de-9d45-7017368061b1 None None] MFA Rules not processed for user `fbc2d50f87044733ba33c2a07c3ede20`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:11:03.077664 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9d2e0c2a-401f-46de-9d45-7017368061b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:11:03.126576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48e3c3-f192-47fc-a0bb-dc2b93becaee None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:03.130416 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa74d07-b846-43fc-a8b2-e8ae8c0f575f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:03.131077 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa74d07-b846-43fc-a8b2-e8ae8c0f575f service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:03.131348 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa74d07-b846-43fc-a8b2-e8ae8c0f575f service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:03.131748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa74d07-b846-43fc-a8b2-e8ae8c0f575f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:03.142464 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa74d07-b846-43fc-a8b2-e8ae8c0f575f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:03.143924 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa74d07-b846-43fc-a8b2-e8ae8c0f575f service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:03.150055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1356/2715] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:11:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:11:03.154860 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1360/2716] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:11:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:11:03.160621 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e0c2a-401f-46de-9d45-7017368061b1 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:03.162000 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9d2e0c2a-401f-46de-9d45-7017368061b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:11:03.218814 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e0c2a-401f-46de-9d45-7017368061b1 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:03.222727 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8addde9-014d-4fa7-a6a2-56fc40c86e8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:03.223368 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8addde9-014d-4fa7-a6a2-56fc40c86e8d service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:03.223650 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8addde9-014d-4fa7-a6a2-56fc40c86e8d service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:03.223924 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8addde9-014d-4fa7-a6a2-56fc40c86e8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:03.232878 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8addde9-014d-4fa7-a6a2-56fc40c86e8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:03.234338 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8addde9-014d-4fa7-a6a2-56fc40c86e8d service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:03.243248 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1357/2717] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:11:03 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:11:06.458169 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdfd83-6158-466a-b28e-c82dba614f67 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:06.459034 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdfd83-6158-466a-b28e-c82dba614f67 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:06.459318 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdfd83-6158-466a-b28e-c82dba614f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:06.476865 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-7afdfd83-6158-466a-b28e-c82dba614f67 None None] MFA Rules not processed for user `1de1f7ce5e5f4cfbb283f0c79d87f9c8`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:11:06.478359 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7afdfd83-6158-466a-b28e-c82dba614f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:11:06.540623 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1361/2718] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:11:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:11:08.774000 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4a2ca-2612-4769-9350-d233537d5043 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:08.774741 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4a2ca-2612-4769-9350-d233537d5043 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:08.774741 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4a2ca-2612-4769-9350-d233537d5043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:08.807231 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-71a4a2ca-2612-4769-9350-d233537d5043 None None] MFA Rules not processed for user `4e2656d339a74bd882c2226251ac3d36`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:11:08.809857 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-71a4a2ca-2612-4769-9350-d233537d5043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:11:08.877479 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1358/2719] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:11:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:11:08.882327 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdfd83-6158-466a-b28e-c82dba614f67 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:08.892676 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdfd83-6158-466a-b28e-c82dba614f67 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:08.897108 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c738a-dae3-4838-80fd-3ddcfd1775d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:08.897749 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c738a-dae3-4838-80fd-3ddcfd1775d4 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:08.898028 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c738a-dae3-4838-80fd-3ddcfd1775d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:08.898335 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c738a-dae3-4838-80fd-3ddcfd1775d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:08.899054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c738a-dae3-4838-80fd-3ddcfd1775d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:08.901483 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c738a-dae3-4838-80fd-3ddcfd1775d4 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:08.925958 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1362/2720] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:11:08 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:11:08.930009 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a4a2ca-2612-4769-9350-d233537d5043 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:08.940625 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a4a2ca-2612-4769-9350-d233537d5043 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:08.995190 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3348e4cc-0ff1-4cf9-83b2-72da6d7687a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:08.995752 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348e4cc-0ff1-4cf9-83b2-72da6d7687a2 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:08.996033 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348e4cc-0ff1-4cf9-83b2-72da6d7687a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:08.996309 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348e4cc-0ff1-4cf9-83b2-72da6d7687a2 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:08.996992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3348e4cc-0ff1-4cf9-83b2-72da6d7687a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:08.998924 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3348e4cc-0ff1-4cf9-83b2-72da6d7687a2 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:09.022724 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1359/2721] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:11:08 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:11:09.027320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c738a-dae3-4838-80fd-3ddcfd1775d4 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:09.038131 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c738a-dae3-4838-80fd-3ddcfd1775d4 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:09.043516 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f60f7-060e-4095-aa58-5180a906dd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:09.044266 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f60f7-060e-4095-aa58-5180a906dd11 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:09.044550 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f60f7-060e-4095-aa58-5180a906dd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:09.044825 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f60f7-060e-4095-aa58-5180a906dd11 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:09.045558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46f60f7-060e-4095-aa58-5180a906dd11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:09.047271 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46f60f7-060e-4095-aa58-5180a906dd11 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:09.058795 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1363/2722] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:11:09.062989 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3348e4cc-0ff1-4cf9-83b2-72da6d7687a2 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:09.073137 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3348e4cc-0ff1-4cf9-83b2-72da6d7687a2 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:09.077644 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5bee9-2869-437f-8cd8-19940a96eb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:09.078548 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5bee9-2869-437f-8cd8-19940a96eb1b admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:09.078830 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5bee9-2869-437f-8cd8-19940a96eb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:09.079101 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5bee9-2869-437f-8cd8-19940a96eb1b admin admin] PATH_INFO: `/v3/projects/96cc6132d1dd4be8aba03b926a5afef7/users/360edf7a5cd4488fac988060b7076405/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:09.093878 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf5bee9-2869-437f-8cd8-19940a96eb1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=96cc6132d1dd4be8aba03b926a5afef7, user_id=360edf7a5cd4488fac988060b7076405, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:09.096559 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf5bee9-2869-437f-8cd8-19940a96eb1b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:09.109717 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1360/2723] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:11:09 2022] PUT /identity/v3/projects/96cc6132d1dd4be8aba03b926a5afef7/users/360edf7a5cd4488fac988060b7076405/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:11:09.113625 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f60f7-060e-4095-aa58-5180a906dd11 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:09.128689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f60f7-060e-4095-aa58-5180a906dd11 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:09.184127 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cdb9c-3ab6-4281-829c-aea872cfa430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:09.185032 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cdb9c-3ab6-4281-829c-aea872cfa430 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:09.185254 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cdb9c-3ab6-4281-829c-aea872cfa430 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:09.185537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cdb9c-3ab6-4281-829c-aea872cfa430 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:09.186290 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0cdb9c-3ab6-4281-829c-aea872cfa430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:09.187867 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0cdb9c-3ab6-4281-829c-aea872cfa430 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:09.201285 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1364/2724] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:11:09.202993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5bee9-2869-437f-8cd8-19940a96eb1b admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:09.214131 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5bee9-2869-437f-8cd8-19940a96eb1b admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:09.219029 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d092ea-419c-4075-b8cc-c241defe5d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:09.219997 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d092ea-419c-4075-b8cc-c241defe5d11 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:09.220356 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d092ea-419c-4075-b8cc-c241defe5d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:09.220693 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d092ea-419c-4075-b8cc-c241defe5d11 admin admin] PATH_INFO: `/v3/projects/96cc6132d1dd4be8aba03b926a5afef7/users/360edf7a5cd4488fac988060b7076405/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:09.222786 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d092ea-419c-4075-b8cc-c241defe5d11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96cc6132d1dd4be8aba03b926a5afef7, user_id=360edf7a5cd4488fac988060b7076405, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:09.225941 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d092ea-419c-4075-b8cc-c241defe5d11 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:09.236943 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1361/2725] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:11:09 2022] PUT /identity/v3/projects/96cc6132d1dd4be8aba03b926a5afef7/users/360edf7a5cd4488fac988060b7076405/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:11:14.472266 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961d423-155d-4737-bbdb-838a646a087b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:14.472902 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961d423-155d-4737-bbdb-838a646a087b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:14.472902 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961d423-155d-4737-bbdb-838a646a087b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:14.503933 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-6961d423-155d-4737-bbdb-838a646a087b None None] MFA Rules not processed for user `360edf7a5cd4488fac988060b7076405`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:11:14.506018 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6961d423-155d-4737-bbdb-838a646a087b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:11:14.585815 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1365/2726] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:11:14 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:11:14.600853 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d092ea-419c-4075-b8cc-c241defe5d11 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:14.602675 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-08d092ea-419c-4075-b8cc-c241defe5d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:11:14.704260 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d092ea-419c-4075-b8cc-c241defe5d11 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:14.714093 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601a4ba-df9c-4d8b-ba67-0c80785bbe83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:14.714093 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601a4ba-df9c-4d8b-ba67-0c80785bbe83 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:14.714093 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601a4ba-df9c-4d8b-ba67-0c80785bbe83 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:14.714093 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601a4ba-df9c-4d8b-ba67-0c80785bbe83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:14.775539 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e601a4ba-df9c-4d8b-ba67-0c80785bbe83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:14.777137 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e601a4ba-df9c-4d8b-ba67-0c80785bbe83 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:14.788837 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1362/2727] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:11:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:11:16.331147 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367d6e7-b429-4238-a86d-3331f3e032a4 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:16.331689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367d6e7-b429-4238-a86d-3331f3e032a4 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:16.331689 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367d6e7-b429-4238-a86d-3331f3e032a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:16.350539 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-9367d6e7-b429-4238-a86d-3331f3e032a4 None None] MFA Rules not processed for user `4e2656d339a74bd882c2226251ac3d36`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:11:16.352013 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9367d6e7-b429-4238-a86d-3331f3e032a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:11:16.360591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1366/2728] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:11:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:11:20.001763 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601a4ba-df9c-4d8b-ba67-0c80785bbe83 service nova] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:20.062986 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601a4ba-df9c-4d8b-ba67-0c80785bbe83 service nova] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:20.067598 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1618a9-bf51-417c-8634-10eea14bf3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:20.068473 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1618a9-bf51-417c-8634-10eea14bf3b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:20.068911 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1618a9-bf51-417c-8634-10eea14bf3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:20.069183 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1618a9-bf51-417c-8634-10eea14bf3b8 admin admin] PATH_INFO: `/v3/users/fbc2d50f87044733ba33c2a07c3ede20` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:20.070468 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1618a9-bf51-417c-8634-10eea14bf3b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbc2d50f87044733ba33c2a07c3ede20)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:20.073028 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1618a9-bf51-417c-8634-10eea14bf3b8 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:20.122357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8d1618a9-bf51-417c-8634-10eea14bf3b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbc2d50f87044733ba33c2a07c3ede20'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:20.128945 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8d1618a9-bf51-417c-8634-10eea14bf3b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbc2d50f87044733ba33c2a07c3ede20'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:20.129658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8d1618a9-bf51-417c-8634-10eea14bf3b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbc2d50f87044733ba33c2a07c3ede20'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:20.138950 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8d1618a9-bf51-417c-8634-10eea14bf3b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbc2d50f87044733ba33c2a07c3ede20'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:20.140294 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8d1618a9-bf51-417c-8634-10eea14bf3b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbc2d50f87044733ba33c2a07c3ede20'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:20.158687 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1363/2729] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:11:20 2022] DELETE /identity/v3/users/fbc2d50f87044733ba33c2a07c3ede20 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:11:20.163494 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8733c-f996-4919-9b71-b1abc492090a None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:20.164032 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8733c-f996-4919-9b71-b1abc492090a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:20.164149 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8733c-f996-4919-9b71-b1abc492090a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:20.182753 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-9ef8733c-f996-4919-9b71-b1abc492090a None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:11:20.183966 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9ef8733c-f996-4919-9b71-b1abc492090a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:11:20.246889 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1367/2730] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:11:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:11:20.256555 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1618a9-bf51-417c-8634-10eea14bf3b8 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:20.257952 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8d1618a9-bf51-417c-8634-10eea14bf3b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:11:20.314313 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1618a9-bf51-417c-8634-10eea14bf3b8 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:20.317887 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c7b88-d600-492d-863d-8033d1f77ec7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:20.318555 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c7b88-d600-492d-863d-8033d1f77ec7 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:20.318834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c7b88-d600-492d-863d-8033d1f77ec7 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:20.319104 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c7b88-d600-492d-863d-8033d1f77ec7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:20.328663 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c7b88-d600-492d-863d-8033d1f77ec7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:20.330258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c7b88-d600-492d-863d-8033d1f77ec7 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:20.340543 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1364/2731] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:11:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:11:20.611960 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8733c-f996-4919-9b71-b1abc492090a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:20.613408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9ef8733c-f996-4919-9b71-b1abc492090a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:11:20.625774 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8733c-f996-4919-9b71-b1abc492090a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:20.640603 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1acc2-624a-4550-9fa3-60f182ac0581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:20.641473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1acc2-624a-4550-9fa3-60f182ac0581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:20.642251 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1acc2-624a-4550-9fa3-60f182ac0581 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:20.642535 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1acc2-624a-4550-9fa3-60f182ac0581 admin admin] PATH_INFO: `/v3/projects/f0a7b934e5744e34b881025ab35a6799` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:20.643765 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1acc2-624a-4550-9fa3-60f182ac0581 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0a7b934e5744e34b881025ab35a6799)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:20.645652 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1acc2-624a-4550-9fa3-60f182ac0581 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:20.699906 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d2d1acc2-624a-4550-9fa3-60f182ac0581 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0a7b934e5744e34b881025ab35a6799', 'request_id': 'req-d2d1acc2-624a-4550-9fa3-60f182ac0581'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:20.705313 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d2d1acc2-624a-4550-9fa3-60f182ac0581 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0a7b934e5744e34b881025ab35a6799', 'request_id': 'req-d2d1acc2-624a-4550-9fa3-60f182ac0581'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:20.711416 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d2d1acc2-624a-4550-9fa3-60f182ac0581 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0a7b934e5744e34b881025ab35a6799', 'request_id': 'req-d2d1acc2-624a-4550-9fa3-60f182ac0581'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:20.726026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d2d1acc2-624a-4550-9fa3-60f182ac0581 admin admin] The token cache is being invalidate because project f0a7b934e5744e34b881025ab35a6799 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:11:20.726426 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d2d1acc2-624a-4550-9fa3-60f182ac0581 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:20.727047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d2d1acc2-624a-4550-9fa3-60f182ac0581 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:11:20.729137 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1368/2732] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:11:20 2022] DELETE /identity/v3/projects/f0a7b934e5744e34b881025ab35a6799 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:11:46.538465 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c7b88-d600-492d-863d-8033d1f77ec7 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:11:46.539848 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-639c7b88-d600-492d-863d-8033d1f77ec7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:11:46.550760 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c7b88-d600-492d-863d-8033d1f77ec7 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:11:46.597114 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b70814-8a94-4da9-9cd3-f4cbdb6ff555 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:11:46.597746 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b70814-8a94-4da9-9cd3-f4cbdb6ff555 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:11:46.598023 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b70814-8a94-4da9-9cd3-f4cbdb6ff555 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:11:46.598319 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b70814-8a94-4da9-9cd3-f4cbdb6ff555 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:11:46.600072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d1b70814-8a94-4da9-9cd3-f4cbdb6ff555 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:11:46.659090 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b70814-8a94-4da9-9cd3-f4cbdb6ff555 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:11:46.660512 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b70814-8a94-4da9-9cd3-f4cbdb6ff555 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:11:46.669629 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1365/2733] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:11:46 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:12:21.997990 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1acc2-624a-4550-9fa3-60f182ac0581 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:21.999478 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d2d1acc2-624a-4550-9fa3-60f182ac0581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:22.011978 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1acc2-624a-4550-9fa3-60f182ac0581 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:22.016139 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a4e17-b3ed-4215-a9a3-01c769ce2b45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:22.016771 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a4e17-b3ed-4215-a9a3-01c769ce2b45 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:22.017054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a4e17-b3ed-4215-a9a3-01c769ce2b45 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:22.017329 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a4e17-b3ed-4215-a9a3-01c769ce2b45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:22.019206 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-817a4e17-b3ed-4215-a9a3-01c769ce2b45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:22.086841 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a4e17-b3ed-4215-a9a3-01c769ce2b45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:22.088256 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a4e17-b3ed-4215-a9a3-01c769ce2b45 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:22.098901 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1369/2734] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:12:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:12:33.677506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b70814-8a94-4da9-9cd3-f4cbdb6ff555 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:33.679055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d1b70814-8a94-4da9-9cd3-f4cbdb6ff555 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:33.755188 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b70814-8a94-4da9-9cd3-f4cbdb6ff555 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:33.759957 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe591858-8436-4a04-98f5-33d22021d77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:33.760822 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe591858-8436-4a04-98f5-33d22021d77c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:33.761182 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe591858-8436-4a04-98f5-33d22021d77c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:33.761354 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe591858-8436-4a04-98f5-33d22021d77c admin admin] PATH_INFO: `/v3/users/1de1f7ce5e5f4cfbb283f0c79d87f9c8` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:33.762589 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe591858-8436-4a04-98f5-33d22021d77c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1de1f7ce5e5f4cfbb283f0c79d87f9c8)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:33.764493 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe591858-8436-4a04-98f5-33d22021d77c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:33.812134 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-fe591858-8436-4a04-98f5-33d22021d77c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1de1f7ce5e5f4cfbb283f0c79d87f9c8'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:33.818774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-fe591858-8436-4a04-98f5-33d22021d77c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1de1f7ce5e5f4cfbb283f0c79d87f9c8'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:33.819452 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-fe591858-8436-4a04-98f5-33d22021d77c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1de1f7ce5e5f4cfbb283f0c79d87f9c8'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:33.826299 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-fe591858-8436-4a04-98f5-33d22021d77c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1de1f7ce5e5f4cfbb283f0c79d87f9c8'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:33.826917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-fe591858-8436-4a04-98f5-33d22021d77c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1de1f7ce5e5f4cfbb283f0c79d87f9c8'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:33.844075 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1366/2735] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:12:33 2022] DELETE /identity/v3/users/1de1f7ce5e5f4cfbb283f0c79d87f9c8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:12:34.016972 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a4e17-b3ed-4215-a9a3-01c769ce2b45 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:34.028027 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a4e17-b3ed-4215-a9a3-01c769ce2b45 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:34.087777 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5467a527-5ced-4c82-b783-8fb02a847549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:34.088624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467a527-5ced-4c82-b783-8fb02a847549 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:34.088899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467a527-5ced-4c82-b783-8fb02a847549 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:34.089167 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467a527-5ced-4c82-b783-8fb02a847549 admin admin] PATH_INFO: `/v3/projects/c0df002b4f614f4bbac1782e1fbb484a` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:34.090340 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5467a527-5ced-4c82-b783-8fb02a847549 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0df002b4f614f4bbac1782e1fbb484a)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:34.092216 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5467a527-5ced-4c82-b783-8fb02a847549 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:34.136184 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-5467a527-5ced-4c82-b783-8fb02a847549 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0df002b4f614f4bbac1782e1fbb484a', 'request_id': 'req-5467a527-5ced-4c82-b783-8fb02a847549'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:34.142169 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-5467a527-5ced-4c82-b783-8fb02a847549 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0df002b4f614f4bbac1782e1fbb484a', 'request_id': 'req-5467a527-5ced-4c82-b783-8fb02a847549'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:34.148047 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-5467a527-5ced-4c82-b783-8fb02a847549 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0df002b4f614f4bbac1782e1fbb484a', 'request_id': 'req-5467a527-5ced-4c82-b783-8fb02a847549'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:34.159418 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-5467a527-5ced-4c82-b783-8fb02a847549 admin admin] The token cache is being invalidate because project c0df002b4f614f4bbac1782e1fbb484a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:12:34.159704 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-5467a527-5ced-4c82-b783-8fb02a847549 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:34.160299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-5467a527-5ced-4c82-b783-8fb02a847549 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:34.162092 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1370/2736] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:12:34 2022] DELETE /identity/v3/projects/c0df002b4f614f4bbac1782e1fbb484a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:12:34.167054 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce906977-0855-42cf-bd82-05376eee9883 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:34.167345 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce906977-0855-42cf-bd82-05376eee9883 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:34.167625 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce906977-0855-42cf-bd82-05376eee9883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:34.184425 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ce906977-0855-42cf-bd82-05376eee9883 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:12:34.185601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ce906977-0855-42cf-bd82-05376eee9883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:34.243015 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1367/2737] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:12:34.250562 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff5946-1390-4903-953c-db2dfc058e8c None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:34.251416 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff5946-1390-4903-953c-db2dfc058e8c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:34.251697 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff5946-1390-4903-953c-db2dfc058e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:34.269691 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-32ff5946-1390-4903-953c-db2dfc058e8c None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:12:34.270925 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-32ff5946-1390-4903-953c-db2dfc058e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:34.279034 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1371/2738] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:12:34.283137 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce906977-0855-42cf-bd82-05376eee9883 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:34.292690 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce906977-0855-42cf-bd82-05376eee9883 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:34.297203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f54e12c-7efc-4023-a640-b7b85c1d0c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:34.298399 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f54e12c-7efc-4023-a640-b7b85c1d0c74 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:34.298673 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f54e12c-7efc-4023-a640-b7b85c1d0c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:34.298940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f54e12c-7efc-4023-a640-b7b85c1d0c74 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:34.299647 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f54e12c-7efc-4023-a640-b7b85c1d0c74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:12:34.299948 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f54e12c-7efc-4023-a640-b7b85c1d0c74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:34.301551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f54e12c-7efc-4023-a640-b7b85c1d0c74 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:34.312825 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1368/2739] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:12:34.317428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff5946-1390-4903-953c-db2dfc058e8c None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:34.326429 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff5946-1390-4903-953c-db2dfc058e8c None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:34.331058 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eac514-9afd-4522-9fa2-6cf656169e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:34.331695 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eac514-9afd-4522-9fa2-6cf656169e45 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:34.331978 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eac514-9afd-4522-9fa2-6cf656169e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:34.332252 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eac514-9afd-4522-9fa2-6cf656169e45 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:34.332942 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eac514-9afd-4522-9fa2-6cf656169e45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:34.334885 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eac514-9afd-4522-9fa2-6cf656169e45 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:34.360723 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1372/2740] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:12:34 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:12:34.364771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f54e12c-7efc-4023-a640-b7b85c1d0c74 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:34.374461 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f54e12c-7efc-4023-a640-b7b85c1d0c74 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:34.427604 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e7f89-3d29-4143-a835-c6e039061e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:34.428243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e7f89-3d29-4143-a835-c6e039061e7d admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:34.428525 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e7f89-3d29-4143-a835-c6e039061e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:34.428796 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e7f89-3d29-4143-a835-c6e039061e7d admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:34.429464 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2e7f89-3d29-4143-a835-c6e039061e7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:34.431390 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2e7f89-3d29-4143-a835-c6e039061e7d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:34.453491 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1369/2741] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:12:34 2022] POST /identity/v3/users => generated 293 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:12:34.457642 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eac514-9afd-4522-9fa2-6cf656169e45 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:34.468142 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eac514-9afd-4522-9fa2-6cf656169e45 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:34.472839 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098a654-a894-40d4-8585-064d3e2c0353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:34.473605 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098a654-a894-40d4-8585-064d3e2c0353 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:34.473892 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098a654-a894-40d4-8585-064d3e2c0353 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:34.474197 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098a654-a894-40d4-8585-064d3e2c0353 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:34.474918 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9098a654-a894-40d4-8585-064d3e2c0353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:34.476510 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9098a654-a894-40d4-8585-064d3e2c0353 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:34.488477 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1373/2742] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:12:34.492302 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e7f89-3d29-4143-a835-c6e039061e7d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:34.502572 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e7f89-3d29-4143-a835-c6e039061e7d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:34.507175 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1474d8-6eac-4540-9503-7431309f3773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:34.507817 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1474d8-6eac-4540-9503-7431309f3773 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:34.508097 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1474d8-6eac-4540-9503-7431309f3773 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:34.508546 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1474d8-6eac-4540-9503-7431309f3773 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:34.509264 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1474d8-6eac-4540-9503-7431309f3773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:34.510896 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1474d8-6eac-4540-9503-7431309f3773 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:34.520529 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1370/2743] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:12:34.524492 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098a654-a894-40d4-8585-064d3e2c0353 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:34.534054 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098a654-a894-40d4-8585-064d3e2c0353 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:34.538861 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00a8b1-dae3-4449-83f5-b24f9d98a69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:34.539744 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00a8b1-dae3-4449-83f5-b24f9d98a69b admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:34.540027 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00a8b1-dae3-4449-83f5-b24f9d98a69b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:34.540303 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00a8b1-dae3-4449-83f5-b24f9d98a69b admin admin] PATH_INFO: `/v3/projects/393d9067cef543b5ad23dc37a24f0ae5/users/e17437ea6d3b44ebbcdcbeb4b6666ad7/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:34.554781 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00a8b1-dae3-4449-83f5-b24f9d98a69b admin admin] RBAC: Authorizing `identity:create_grant(project_id=393d9067cef543b5ad23dc37a24f0ae5, user_id=e17437ea6d3b44ebbcdcbeb4b6666ad7, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:34.557416 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00a8b1-dae3-4449-83f5-b24f9d98a69b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:34.570240 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1374/2744] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:12:34 2022] PUT /identity/v3/projects/393d9067cef543b5ad23dc37a24f0ae5/users/e17437ea6d3b44ebbcdcbeb4b6666ad7/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:12:34.581154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13950d-4bc5-4beb-95a5-cd752a08bf9f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:34.581445 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13950d-4bc5-4beb-95a5-cd752a08bf9f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:34.581748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13950d-4bc5-4beb-95a5-cd752a08bf9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:34.607408 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-7b13950d-4bc5-4beb-95a5-cd752a08bf9f None None] MFA Rules not processed for user `e17437ea6d3b44ebbcdcbeb4b6666ad7`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:12:34.608592 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7b13950d-4bc5-4beb-95a5-cd752a08bf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:34.670435 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1371/2745] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:12:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:12:34.681991 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00a8b1-dae3-4449-83f5-b24f9d98a69b admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:34.683684 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3c00a8b1-dae3-4449-83f5-b24f9d98a69b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:34.754158 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00a8b1-dae3-4449-83f5-b24f9d98a69b admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:34.759556 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87e418-e323-4a4f-8976-ba092c6f787c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:34.760191 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87e418-e323-4a4f-8976-ba092c6f787c service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:34.760468 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87e418-e323-4a4f-8976-ba092c6f787c service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:34.760741 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87e418-e323-4a4f-8976-ba092c6f787c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:34.768714 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f87e418-e323-4a4f-8976-ba092c6f787c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:34.770314 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f87e418-e323-4a4f-8976-ba092c6f787c service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:34.778971 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1375/2746] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:12:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:12:34.799974 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13950d-4bc5-4beb-95a5-cd752a08bf9f None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:34.801380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7b13950d-4bc5-4beb-95a5-cd752a08bf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:34.856149 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13950d-4bc5-4beb-95a5-cd752a08bf9f None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:34.860056 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7e1ac-aa3f-4ab5-a3ae-b6b8dfacfa03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:34.861194 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7e1ac-aa3f-4ab5-a3ae-b6b8dfacfa03 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:34.861453 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7e1ac-aa3f-4ab5-a3ae-b6b8dfacfa03 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:34.861734 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7e1ac-aa3f-4ab5-a3ae-b6b8dfacfa03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:34.863545 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1ee7e1ac-aa3f-4ab5-a3ae-b6b8dfacfa03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:34.876711 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7e1ac-aa3f-4ab5-a3ae-b6b8dfacfa03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:34.878131 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7e1ac-aa3f-4ab5-a3ae-b6b8dfacfa03 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:34.888647 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1372/2747] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:12:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:12:42.084695 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87e418-e323-4a4f-8976-ba092c6f787c service nova] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:42.141704 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87e418-e323-4a4f-8976-ba092c6f787c service nova] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:42.146188 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e549b6-2cc6-4126-8883-a71af70faaf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:42.147038 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e549b6-2cc6-4126-8883-a71af70faaf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:42.147316 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e549b6-2cc6-4126-8883-a71af70faaf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:42.147590 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e549b6-2cc6-4126-8883-a71af70faaf8 admin admin] PATH_INFO: `/v3/users/e17437ea6d3b44ebbcdcbeb4b6666ad7` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:42.148751 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e549b6-2cc6-4126-8883-a71af70faaf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e17437ea6d3b44ebbcdcbeb4b6666ad7)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:42.150679 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e549b6-2cc6-4126-8883-a71af70faaf8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:42.195895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e3e549b6-2cc6-4126-8883-a71af70faaf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e17437ea6d3b44ebbcdcbeb4b6666ad7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:42.202578 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e3e549b6-2cc6-4126-8883-a71af70faaf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e17437ea6d3b44ebbcdcbeb4b6666ad7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:42.209462 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e3e549b6-2cc6-4126-8883-a71af70faaf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e17437ea6d3b44ebbcdcbeb4b6666ad7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:42.210078 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e3e549b6-2cc6-4126-8883-a71af70faaf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e17437ea6d3b44ebbcdcbeb4b6666ad7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:42.222063 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e3e549b6-2cc6-4126-8883-a71af70faaf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e17437ea6d3b44ebbcdcbeb4b6666ad7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:42.227789 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1376/2748] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:12:42 2022] DELETE /identity/v3/users/e17437ea6d3b44ebbcdcbeb4b6666ad7 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:12:42.232510 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcb8c4-d649-4d66-9d3b-52a7bb48bbcd None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:42.232975 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcb8c4-d649-4d66-9d3b-52a7bb48bbcd None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:42.233105 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcb8c4-d649-4d66-9d3b-52a7bb48bbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:42.251798 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b4bcb8c4-d649-4d66-9d3b-52a7bb48bbcd None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:12:42.253024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b4bcb8c4-d649-4d66-9d3b-52a7bb48bbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:42.308321 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1373/2749] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:12:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:12:42.317733 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e549b6-2cc6-4126-8883-a71af70faaf8 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:42.319199 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e3e549b6-2cc6-4126-8883-a71af70faaf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:42.374908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e549b6-2cc6-4126-8883-a71af70faaf8 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:42.378560 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f595644-673f-485e-8d95-9fd4ff28d1ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:42.379209 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f595644-673f-485e-8d95-9fd4ff28d1ba service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:42.379477 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f595644-673f-485e-8d95-9fd4ff28d1ba service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:42.379749 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f595644-673f-485e-8d95-9fd4ff28d1ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:42.388954 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f595644-673f-485e-8d95-9fd4ff28d1ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:42.390395 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f595644-673f-485e-8d95-9fd4ff28d1ba service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:42.399570 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1377/2750] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:12:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:12:42.660460 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bcb8c4-d649-4d66-9d3b-52a7bb48bbcd None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:42.661895 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b4bcb8c4-d649-4d66-9d3b-52a7bb48bbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:42.673686 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bcb8c4-d649-4d66-9d3b-52a7bb48bbcd None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:42.683469 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782e73a-dfaa-4720-8324-86101b520517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:42.684329 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782e73a-dfaa-4720-8324-86101b520517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:42.684767 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782e73a-dfaa-4720-8324-86101b520517 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:42.685075 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782e73a-dfaa-4720-8324-86101b520517 admin admin] PATH_INFO: `/v3/projects/393d9067cef543b5ad23dc37a24f0ae5` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:42.686269 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782e73a-dfaa-4720-8324-86101b520517 admin admin] RBAC: Authorizing `identity:delete_project(project_id=393d9067cef543b5ad23dc37a24f0ae5)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:42.688190 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782e73a-dfaa-4720-8324-86101b520517 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:42.733168 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-6782e73a-dfaa-4720-8324-86101b520517 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '393d9067cef543b5ad23dc37a24f0ae5', 'request_id': 'req-6782e73a-dfaa-4720-8324-86101b520517'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:42.739050 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-6782e73a-dfaa-4720-8324-86101b520517 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '393d9067cef543b5ad23dc37a24f0ae5', 'request_id': 'req-6782e73a-dfaa-4720-8324-86101b520517'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:42.744250 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-6782e73a-dfaa-4720-8324-86101b520517 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '393d9067cef543b5ad23dc37a24f0ae5', 'request_id': 'req-6782e73a-dfaa-4720-8324-86101b520517'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:42.755834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-6782e73a-dfaa-4720-8324-86101b520517 admin admin] The token cache is being invalidate because project 393d9067cef543b5ad23dc37a24f0ae5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:12:42.756125 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-6782e73a-dfaa-4720-8324-86101b520517 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:42.756741 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-6782e73a-dfaa-4720-8324-86101b520517 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:42.758308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1374/2751] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:12:42 2022] DELETE /identity/v3/projects/393d9067cef543b5ad23dc37a24f0ae5 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:12:42.763721 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee952c0a-4080-4676-b669-2f6d03edc47c None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:42.764034 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee952c0a-4080-4676-b669-2f6d03edc47c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:42.764312 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee952c0a-4080-4676-b669-2f6d03edc47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:42.782417 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-ee952c0a-4080-4676-b669-2f6d03edc47c None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:12:42.783633 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ee952c0a-4080-4676-b669-2f6d03edc47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:42.838300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1378/2752] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:12:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:12:42.846332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309794a0-cb1f-4dc3-8390-2d902c6d095b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:42.846618 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309794a0-cb1f-4dc3-8390-2d902c6d095b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:42.846887 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309794a0-cb1f-4dc3-8390-2d902c6d095b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:42.870438 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-309794a0-cb1f-4dc3-8390-2d902c6d095b None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:12:42.871644 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-309794a0-cb1f-4dc3-8390-2d902c6d095b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:42.880369 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1375/2753] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:12:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:12:42.884654 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee952c0a-4080-4676-b669-2f6d03edc47c None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:42.894837 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee952c0a-4080-4676-b669-2f6d03edc47c None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:42.899339 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab680c-2245-485a-b890-8df3353cc631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:42.899978 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab680c-2245-485a-b890-8df3353cc631 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:42.900260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab680c-2245-485a-b890-8df3353cc631 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:42.900532 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab680c-2245-485a-b890-8df3353cc631 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:42.901247 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab680c-2245-485a-b890-8df3353cc631 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:12:42.901551 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab680c-2245-485a-b890-8df3353cc631 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:42.903154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab680c-2245-485a-b890-8df3353cc631 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:42.914428 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1379/2754] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:12:42.918380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309794a0-cb1f-4dc3-8390-2d902c6d095b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:42.927429 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309794a0-cb1f-4dc3-8390-2d902c6d095b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:42.932071 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699c777-0b02-41cf-9944-a145cc2069b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:42.932710 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699c777-0b02-41cf-9944-a145cc2069b6 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:42.932988 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699c777-0b02-41cf-9944-a145cc2069b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:42.933258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699c777-0b02-41cf-9944-a145cc2069b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:42.933947 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699c777-0b02-41cf-9944-a145cc2069b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:42.935885 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699c777-0b02-41cf-9944-a145cc2069b6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:42.961465 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1376/2755] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:12:42 2022] POST /identity/v3/projects => generated 371 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:12:42.965244 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab680c-2245-485a-b890-8df3353cc631 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:42.975146 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab680c-2245-485a-b890-8df3353cc631 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:43.030199 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50f72d-93a9-4bf1-8911-c26e35c21cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:43.030864 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50f72d-93a9-4bf1-8911-c26e35c21cfc admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:43.031141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50f72d-93a9-4bf1-8911-c26e35c21cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:43.031441 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50f72d-93a9-4bf1-8911-c26e35c21cfc admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:43.032193 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50f72d-93a9-4bf1-8911-c26e35c21cfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:43.034155 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50f72d-93a9-4bf1-8911-c26e35c21cfc admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:43.054579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1380/2756] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:12:42 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:12:43.058824 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699c777-0b02-41cf-9944-a145cc2069b6 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:43.069548 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699c777-0b02-41cf-9944-a145cc2069b6 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:43.074327 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10310ba-b721-4673-91e9-d5d8e1e69e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:43.074978 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10310ba-b721-4673-91e9-d5d8e1e69e4c admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:43.075256 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10310ba-b721-4673-91e9-d5d8e1e69e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:43.075527 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10310ba-b721-4673-91e9-d5d8e1e69e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:43.076236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10310ba-b721-4673-91e9-d5d8e1e69e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:43.077825 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10310ba-b721-4673-91e9-d5d8e1e69e4c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:43.088556 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1377/2757] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:12:43.092299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50f72d-93a9-4bf1-8911-c26e35c21cfc admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:43.102241 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50f72d-93a9-4bf1-8911-c26e35c21cfc admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:43.107115 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5117f8e3-0d05-4470-a62e-721e426e0f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:43.107765 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5117f8e3-0d05-4470-a62e-721e426e0f91 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:43.108304 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5117f8e3-0d05-4470-a62e-721e426e0f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:43.108304 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5117f8e3-0d05-4470-a62e-721e426e0f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:43.109023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5117f8e3-0d05-4470-a62e-721e426e0f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:43.110776 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5117f8e3-0d05-4470-a62e-721e426e0f91 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:43.120416 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1381/2758] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:12:43.124365 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10310ba-b721-4673-91e9-d5d8e1e69e4c admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:43.134611 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10310ba-b721-4673-91e9-d5d8e1e69e4c admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:43.139188 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73146f-658e-47fb-b7e9-8aede6dc84e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:43.140053 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73146f-658e-47fb-b7e9-8aede6dc84e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:43.140329 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73146f-658e-47fb-b7e9-8aede6dc84e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:43.140598 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73146f-658e-47fb-b7e9-8aede6dc84e2 admin admin] PATH_INFO: `/v3/projects/3a940ddf496a432ca64efb0a01280aed/users/c10fb91235c6458b88815ac7a5904637/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:43.157077 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73146f-658e-47fb-b7e9-8aede6dc84e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a940ddf496a432ca64efb0a01280aed, user_id=c10fb91235c6458b88815ac7a5904637, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:43.159855 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73146f-658e-47fb-b7e9-8aede6dc84e2 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:43.172134 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1378/2759] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:12:43 2022] PUT /identity/v3/projects/3a940ddf496a432ca64efb0a01280aed/users/c10fb91235c6458b88815ac7a5904637/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:12:43.182411 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79359d1-d88f-49aa-ab9b-7df9223f2a72 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:43.182699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79359d1-d88f-49aa-ab9b-7df9223f2a72 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:43.182976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79359d1-d88f-49aa-ab9b-7df9223f2a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:43.210908 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-d79359d1-d88f-49aa-ab9b-7df9223f2a72 None None] MFA Rules not processed for user `c10fb91235c6458b88815ac7a5904637`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:12:43.212097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d79359d1-d88f-49aa-ab9b-7df9223f2a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:43.276089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1382/2760] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:12:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:12:43.287867 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73146f-658e-47fb-b7e9-8aede6dc84e2 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:43.289496 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2d73146f-658e-47fb-b7e9-8aede6dc84e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:43.361393 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73146f-658e-47fb-b7e9-8aede6dc84e2 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:43.367529 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffde361-76f3-4bb2-a033-26be2a1edc5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:43.368229 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffde361-76f3-4bb2-a033-26be2a1edc5a service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:43.368504 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffde361-76f3-4bb2-a033-26be2a1edc5a service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:43.368774 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffde361-76f3-4bb2-a033-26be2a1edc5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:43.377077 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffde361-76f3-4bb2-a033-26be2a1edc5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:43.378667 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffde361-76f3-4bb2-a033-26be2a1edc5a service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:43.387802 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1379/2761] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:12:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:12:46.040009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79359d1-d88f-49aa-ab9b-7df9223f2a72 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:46.098708 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79359d1-d88f-49aa-ab9b-7df9223f2a72 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:46.103107 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61999e57-6f14-47c3-aac4-91daedf8a48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:46.103959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61999e57-6f14-47c3-aac4-91daedf8a48c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:46.104235 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61999e57-6f14-47c3-aac4-91daedf8a48c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:46.104504 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61999e57-6f14-47c3-aac4-91daedf8a48c admin admin] PATH_INFO: `/v3/users/c10fb91235c6458b88815ac7a5904637` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:46.105918 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61999e57-6f14-47c3-aac4-91daedf8a48c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c10fb91235c6458b88815ac7a5904637)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:46.107873 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61999e57-6f14-47c3-aac4-91daedf8a48c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:46.152432 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-61999e57-6f14-47c3-aac4-91daedf8a48c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c10fb91235c6458b88815ac7a5904637'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:46.159710 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-61999e57-6f14-47c3-aac4-91daedf8a48c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c10fb91235c6458b88815ac7a5904637'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:46.166662 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-61999e57-6f14-47c3-aac4-91daedf8a48c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c10fb91235c6458b88815ac7a5904637'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:46.167276 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-61999e57-6f14-47c3-aac4-91daedf8a48c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c10fb91235c6458b88815ac7a5904637'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:46.178907 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-61999e57-6f14-47c3-aac4-91daedf8a48c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c10fb91235c6458b88815ac7a5904637'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:46.185788 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1383/2762] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:12:46 2022] DELETE /identity/v3/users/c10fb91235c6458b88815ac7a5904637 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:12:46.191762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86afd7-7ce3-47bb-80c4-0c7c8522fa81 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:46.192265 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86afd7-7ce3-47bb-80c4-0c7c8522fa81 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:46.192387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86afd7-7ce3-47bb-80c4-0c7c8522fa81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:46.209595 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-9f86afd7-7ce3-47bb-80c4-0c7c8522fa81 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:12:46.210816 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9f86afd7-7ce3-47bb-80c4-0c7c8522fa81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:46.267093 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1380/2763] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:12:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:12:46.276591 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61999e57-6f14-47c3-aac4-91daedf8a48c admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:46.278000 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-61999e57-6f14-47c3-aac4-91daedf8a48c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:46.334434 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61999e57-6f14-47c3-aac4-91daedf8a48c admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:46.338029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ae94c-7220-4c0c-9ce7-22aba5f1fe6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:46.338687 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ae94c-7220-4c0c-9ce7-22aba5f1fe6c service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:46.338968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ae94c-7220-4c0c-9ce7-22aba5f1fe6c service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:46.339270 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ae94c-7220-4c0c-9ce7-22aba5f1fe6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:46.348228 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ae94c-7220-4c0c-9ce7-22aba5f1fe6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:46.349648 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ae94c-7220-4c0c-9ce7-22aba5f1fe6c service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:46.359187 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1384/2764] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:12:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:12:46.615253 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86afd7-7ce3-47bb-80c4-0c7c8522fa81 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:46.616787 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9f86afd7-7ce3-47bb-80c4-0c7c8522fa81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:46.635490 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86afd7-7ce3-47bb-80c4-0c7c8522fa81 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:46.640728 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32152f8a-5ac9-4f92-80f5-03af8e595d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:46.642413 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32152f8a-5ac9-4f92-80f5-03af8e595d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:46.642692 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32152f8a-5ac9-4f92-80f5-03af8e595d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:46.643054 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32152f8a-5ac9-4f92-80f5-03af8e595d31 admin admin] PATH_INFO: `/v3/projects/3a940ddf496a432ca64efb0a01280aed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:46.644617 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32152f8a-5ac9-4f92-80f5-03af8e595d31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a940ddf496a432ca64efb0a01280aed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:46.646540 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32152f8a-5ac9-4f92-80f5-03af8e595d31 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:46.689845 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-32152f8a-5ac9-4f92-80f5-03af8e595d31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a940ddf496a432ca64efb0a01280aed', 'request_id': 'req-32152f8a-5ac9-4f92-80f5-03af8e595d31'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:46.695649 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-32152f8a-5ac9-4f92-80f5-03af8e595d31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a940ddf496a432ca64efb0a01280aed', 'request_id': 'req-32152f8a-5ac9-4f92-80f5-03af8e595d31'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:46.700766 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-32152f8a-5ac9-4f92-80f5-03af8e595d31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a940ddf496a432ca64efb0a01280aed', 'request_id': 'req-32152f8a-5ac9-4f92-80f5-03af8e595d31'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:46.712410 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-32152f8a-5ac9-4f92-80f5-03af8e595d31 admin admin] The token cache is being invalidate because project 3a940ddf496a432ca64efb0a01280aed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:12:46.712691 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-32152f8a-5ac9-4f92-80f5-03af8e595d31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:46.713298 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-32152f8a-5ac9-4f92-80f5-03af8e595d31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:46.714841 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1381/2765] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:12:46 2022] DELETE /identity/v3/projects/3a940ddf496a432ca64efb0a01280aed => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:12:46.719738 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d84c8-2876-4799-9b5f-c14485fa9f60 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:46.720048 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d84c8-2876-4799-9b5f-c14485fa9f60 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:46.720329 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d84c8-2876-4799-9b5f-c14485fa9f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:46.737693 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5f4d84c8-2876-4799-9b5f-c14485fa9f60 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:12:46.739068 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5f4d84c8-2876-4799-9b5f-c14485fa9f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:46.796287 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1385/2766] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:12:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:12:46.804856 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc27bfa-ebe6-4f78-97df-4a2ad772b838 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:46.805139 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc27bfa-ebe6-4f78-97df-4a2ad772b838 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:46.805412 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc27bfa-ebe6-4f78-97df-4a2ad772b838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:46.823401 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-3fc27bfa-ebe6-4f78-97df-4a2ad772b838 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:12:46.824563 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3fc27bfa-ebe6-4f78-97df-4a2ad772b838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:46.832836 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1382/2767] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:12:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:12:46.836768 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d84c8-2876-4799-9b5f-c14485fa9f60 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:46.846201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d84c8-2876-4799-9b5f-c14485fa9f60 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:46.850673 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329800d-1b66-4f00-8672-4563a4c42fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:46.851308 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329800d-1b66-4f00-8672-4563a4c42fe3 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:46.851589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329800d-1b66-4f00-8672-4563a4c42fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:46.851861 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329800d-1b66-4f00-8672-4563a4c42fe3 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:46.852572 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a329800d-1b66-4f00-8672-4563a4c42fe3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:12:46.852875 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a329800d-1b66-4f00-8672-4563a4c42fe3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:46.854529 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a329800d-1b66-4f00-8672-4563a4c42fe3 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:46.866806 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1386/2768] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:12:46.870879 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc27bfa-ebe6-4f78-97df-4a2ad772b838 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:46.880414 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc27bfa-ebe6-4f78-97df-4a2ad772b838 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:46.885038 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082d34d4-5513-46d2-9f58-dfb46921c781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:46.885676 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d34d4-5513-46d2-9f58-dfb46921c781 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:46.885982 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d34d4-5513-46d2-9f58-dfb46921c781 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:46.886307 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d34d4-5513-46d2-9f58-dfb46921c781 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:46.886994 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082d34d4-5513-46d2-9f58-dfb46921c781 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:46.888868 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082d34d4-5513-46d2-9f58-dfb46921c781 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:46.915313 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1383/2769] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:12:46 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:12:46.918902 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329800d-1b66-4f00-8672-4563a4c42fe3 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:46.928639 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329800d-1b66-4f00-8672-4563a4c42fe3 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:46.983598 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8f29e-b3c7-44aa-a073-f087016d8855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:46.984246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8f29e-b3c7-44aa-a073-f087016d8855 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:46.984606 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8f29e-b3c7-44aa-a073-f087016d8855 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:46.985382 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8f29e-b3c7-44aa-a073-f087016d8855 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:46.986057 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8f29e-b3c7-44aa-a073-f087016d8855 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:46.987961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8f29e-b3c7-44aa-a073-f087016d8855 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:47.006646 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1387/2770] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:12:46 2022] POST /identity/v3/users => generated 288 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:12:47.010675 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082d34d4-5513-46d2-9f58-dfb46921c781 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:47.021380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082d34d4-5513-46d2-9f58-dfb46921c781 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:47.026297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43119e0-82f7-4fbf-afcc-ae7f5ba6be6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:47.026974 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43119e0-82f7-4fbf-afcc-ae7f5ba6be6c admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:47.027252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43119e0-82f7-4fbf-afcc-ae7f5ba6be6c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:47.027547 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43119e0-82f7-4fbf-afcc-ae7f5ba6be6c admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:47.028293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43119e0-82f7-4fbf-afcc-ae7f5ba6be6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:47.029874 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43119e0-82f7-4fbf-afcc-ae7f5ba6be6c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:47.041542 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1384/2771] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:12:47.045819 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8f29e-b3c7-44aa-a073-f087016d8855 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:47.055747 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8f29e-b3c7-44aa-a073-f087016d8855 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:47.060332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f6c66-fd08-4914-a229-f3fa87af4999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:47.060980 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f6c66-fd08-4914-a229-f3fa87af4999 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:47.061272 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f6c66-fd08-4914-a229-f3fa87af4999 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:47.061542 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f6c66-fd08-4914-a229-f3fa87af4999 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:47.062301 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730f6c66-fd08-4914-a229-f3fa87af4999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:47.064735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730f6c66-fd08-4914-a229-f3fa87af4999 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:47.075910 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1388/2772] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:12:47.080543 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43119e0-82f7-4fbf-afcc-ae7f5ba6be6c admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:47.090296 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43119e0-82f7-4fbf-afcc-ae7f5ba6be6c admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:47.095516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309f5e5-769f-4cd2-bad3-2f9a72aa0979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:47.096373 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309f5e5-769f-4cd2-bad3-2f9a72aa0979 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:47.096665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309f5e5-769f-4cd2-bad3-2f9a72aa0979 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:47.096928 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309f5e5-769f-4cd2-bad3-2f9a72aa0979 admin admin] PATH_INFO: `/v3/projects/91d8f714d9ef4fecad2300b79b4cf1c3/users/0e792f156e7b483ebd1dacc6882e6303/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:47.111855 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309f5e5-769f-4cd2-bad3-2f9a72aa0979 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91d8f714d9ef4fecad2300b79b4cf1c3, user_id=0e792f156e7b483ebd1dacc6882e6303, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:47.115362 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309f5e5-769f-4cd2-bad3-2f9a72aa0979 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:47.128593 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1385/2773] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:12:47 2022] PUT /identity/v3/projects/91d8f714d9ef4fecad2300b79b4cf1c3/users/0e792f156e7b483ebd1dacc6882e6303/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:12:47.138686 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e78a70-476b-4c39-8107-3a4a7a5c4fbf None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:47.139002 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e78a70-476b-4c39-8107-3a4a7a5c4fbf None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:47.139286 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e78a70-476b-4c39-8107-3a4a7a5c4fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:47.166762 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f1e78a70-476b-4c39-8107-3a4a7a5c4fbf None None] MFA Rules not processed for user `0e792f156e7b483ebd1dacc6882e6303`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:12:47.167968 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f1e78a70-476b-4c39-8107-3a4a7a5c4fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:47.236310 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1389/2774] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:12:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:12:47.247427 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309f5e5-769f-4cd2-bad3-2f9a72aa0979 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:47.249920 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2309f5e5-769f-4cd2-bad3-2f9a72aa0979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:47.320993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309f5e5-769f-4cd2-bad3-2f9a72aa0979 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:47.326420 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933f8eb-7a46-42ec-8979-700b8bf61320 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:47.327056 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933f8eb-7a46-42ec-8979-700b8bf61320 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:47.327335 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933f8eb-7a46-42ec-8979-700b8bf61320 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:47.327607 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933f8eb-7a46-42ec-8979-700b8bf61320 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:47.335499 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933f8eb-7a46-42ec-8979-700b8bf61320 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:47.337087 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933f8eb-7a46-42ec-8979-700b8bf61320 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:47.345990 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1386/2775] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:12:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:12:47.539444 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e78a70-476b-4c39-8107-3a4a7a5c4fbf None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:47.599872 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e78a70-476b-4c39-8107-3a4a7a5c4fbf None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:47.604371 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987ae15-9b30-42c0-9799-35454cca2ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:47.605497 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987ae15-9b30-42c0-9799-35454cca2ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:47.605802 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987ae15-9b30-42c0-9799-35454cca2ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:47.606101 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987ae15-9b30-42c0-9799-35454cca2ffe admin admin] PATH_INFO: `/v3/users/0e792f156e7b483ebd1dacc6882e6303` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:47.607305 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6987ae15-9b30-42c0-9799-35454cca2ffe admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e792f156e7b483ebd1dacc6882e6303)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:47.609219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6987ae15-9b30-42c0-9799-35454cca2ffe admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:47.653844 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6987ae15-9b30-42c0-9799-35454cca2ffe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e792f156e7b483ebd1dacc6882e6303'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:47.660473 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6987ae15-9b30-42c0-9799-35454cca2ffe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e792f156e7b483ebd1dacc6882e6303'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:47.667793 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6987ae15-9b30-42c0-9799-35454cca2ffe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e792f156e7b483ebd1dacc6882e6303'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:47.668523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6987ae15-9b30-42c0-9799-35454cca2ffe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e792f156e7b483ebd1dacc6882e6303'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:47.680235 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6987ae15-9b30-42c0-9799-35454cca2ffe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e792f156e7b483ebd1dacc6882e6303'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:47.685910 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1390/2776] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:12:47 2022] DELETE /identity/v3/users/0e792f156e7b483ebd1dacc6882e6303 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:12:47.691031 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1e969-53ba-4a3a-a481-1ff625e01d41 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:47.691321 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1e969-53ba-4a3a-a481-1ff625e01d41 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:47.691599 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1e969-53ba-4a3a-a481-1ff625e01d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:47.708614 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-6ff1e969-53ba-4a3a-a481-1ff625e01d41 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:12:47.709796 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-6ff1e969-53ba-4a3a-a481-1ff625e01d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:47.765762 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1387/2777] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:12:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:12:47.775307 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987ae15-9b30-42c0-9799-35454cca2ffe admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:47.776709 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6987ae15-9b30-42c0-9799-35454cca2ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:47.831594 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987ae15-9b30-42c0-9799-35454cca2ffe admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:47.835197 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5867f-f10f-4c9d-97eb-e9e32bb501f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:47.835831 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5867f-f10f-4c9d-97eb-e9e32bb501f0 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:47.836111 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5867f-f10f-4c9d-97eb-e9e32bb501f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:47.836382 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5867f-f10f-4c9d-97eb-e9e32bb501f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:47.845269 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f5867f-f10f-4c9d-97eb-e9e32bb501f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:47.846659 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f5867f-f10f-4c9d-97eb-e9e32bb501f0 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:47.856260 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1391/2778] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:12:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:12:48.112509 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1e969-53ba-4a3a-a481-1ff625e01d41 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:48.113942 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-6ff1e969-53ba-4a3a-a481-1ff625e01d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:48.125993 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1e969-53ba-4a3a-a481-1ff625e01d41 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:48.130518 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737caeb8-ad25-4741-aac2-7bc4055bc075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:48.131365 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737caeb8-ad25-4741-aac2-7bc4055bc075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:48.131634 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737caeb8-ad25-4741-aac2-7bc4055bc075 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:48.131897 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737caeb8-ad25-4741-aac2-7bc4055bc075 admin admin] PATH_INFO: `/v3/projects/91d8f714d9ef4fecad2300b79b4cf1c3` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:48.133071 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737caeb8-ad25-4741-aac2-7bc4055bc075 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91d8f714d9ef4fecad2300b79b4cf1c3)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:48.135007 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737caeb8-ad25-4741-aac2-7bc4055bc075 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:48.179971 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-737caeb8-ad25-4741-aac2-7bc4055bc075 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91d8f714d9ef4fecad2300b79b4cf1c3', 'request_id': 'req-737caeb8-ad25-4741-aac2-7bc4055bc075'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:48.185720 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-737caeb8-ad25-4741-aac2-7bc4055bc075 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91d8f714d9ef4fecad2300b79b4cf1c3', 'request_id': 'req-737caeb8-ad25-4741-aac2-7bc4055bc075'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:48.190919 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-737caeb8-ad25-4741-aac2-7bc4055bc075 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91d8f714d9ef4fecad2300b79b4cf1c3', 'request_id': 'req-737caeb8-ad25-4741-aac2-7bc4055bc075'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:48.202390 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-737caeb8-ad25-4741-aac2-7bc4055bc075 admin admin] The token cache is being invalidate because project 91d8f714d9ef4fecad2300b79b4cf1c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:12:48.202670 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-737caeb8-ad25-4741-aac2-7bc4055bc075 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:48.203302 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-737caeb8-ad25-4741-aac2-7bc4055bc075 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:48.204813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1388/2779] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:12:48 2022] DELETE /identity/v3/projects/91d8f714d9ef4fecad2300b79b4cf1c3 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:12:48.209780 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e0c82-c495-4d0f-9da7-c7bc4db740d6 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:48.210281 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e0c82-c495-4d0f-9da7-c7bc4db740d6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:48.210469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e0c82-c495-4d0f-9da7-c7bc4db740d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:48.227951 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-a08e0c82-c495-4d0f-9da7-c7bc4db740d6 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:12:48.229138 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a08e0c82-c495-4d0f-9da7-c7bc4db740d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:48.285446 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1392/2780] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:12:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:12:48.293201 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3ddfc-1867-4c77-8fce-5ad8523f0edf None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:48.293692 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3ddfc-1867-4c77-8fce-5ad8523f0edf None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:48.293804 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3ddfc-1867-4c77-8fce-5ad8523f0edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:48.311745 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-43d3ddfc-1867-4c77-8fce-5ad8523f0edf None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:12:48.312928 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-43d3ddfc-1867-4c77-8fce-5ad8523f0edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:48.320788 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1389/2781] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:12:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:12:48.324777 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e0c82-c495-4d0f-9da7-c7bc4db740d6 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:48.334786 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e0c82-c495-4d0f-9da7-c7bc4db740d6 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:48.339390 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a55b0-a7c1-4d61-bbdb-43991644117f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:48.340029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a55b0-a7c1-4d61-bbdb-43991644117f admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:48.340309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a55b0-a7c1-4d61-bbdb-43991644117f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:48.340583 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a55b0-a7c1-4d61-bbdb-43991644117f admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:48.341325 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a55b0-a7c1-4d61-bbdb-43991644117f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:12:48.341628 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a55b0-a7c1-4d61-bbdb-43991644117f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:48.343259 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a55b0-a7c1-4d61-bbdb-43991644117f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:48.355566 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1393/2782] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:12:48.360530 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3ddfc-1867-4c77-8fce-5ad8523f0edf None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:48.369610 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3ddfc-1867-4c77-8fce-5ad8523f0edf None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:48.374463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af033248-cc69-419d-b384-bd181b79cf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:48.375155 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af033248-cc69-419d-b384-bd181b79cf85 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:48.375431 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af033248-cc69-419d-b384-bd181b79cf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:48.375722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af033248-cc69-419d-b384-bd181b79cf85 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:48.376396 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af033248-cc69-419d-b384-bd181b79cf85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:48.378328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af033248-cc69-419d-b384-bd181b79cf85 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:48.402185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1390/2783] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:12:48 2022] POST /identity/v3/projects => generated 365 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:12:48.406049 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a55b0-a7c1-4d61-bbdb-43991644117f admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:48.416281 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a55b0-a7c1-4d61-bbdb-43991644117f admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:48.470485 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cf62b-2426-4662-bd15-0134356d7efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:48.471141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cf62b-2426-4662-bd15-0134356d7efb admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:48.471425 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cf62b-2426-4662-bd15-0134356d7efb admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:48.471699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cf62b-2426-4662-bd15-0134356d7efb admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:48.472369 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34cf62b-2426-4662-bd15-0134356d7efb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:48.474315 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34cf62b-2426-4662-bd15-0134356d7efb admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:48.496277 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1394/2784] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:12:48 2022] POST /identity/v3/users => generated 283 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:12:48.497585 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af033248-cc69-419d-b384-bd181b79cf85 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:48.507849 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af033248-cc69-419d-b384-bd181b79cf85 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:48.512471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c3e9a-e51a-4800-8337-9b8b188702d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:48.513118 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c3e9a-e51a-4800-8337-9b8b188702d2 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:48.513398 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c3e9a-e51a-4800-8337-9b8b188702d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:48.513670 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c3e9a-e51a-4800-8337-9b8b188702d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:48.514457 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c3e9a-e51a-4800-8337-9b8b188702d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:48.516555 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c3e9a-e51a-4800-8337-9b8b188702d2 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:48.527876 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1391/2785] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:12:48.532456 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cf62b-2426-4662-bd15-0134356d7efb admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:48.542220 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cf62b-2426-4662-bd15-0134356d7efb admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:48.546805 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d219da5-032c-4295-afaf-9953a6b45ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:48.547529 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d219da5-032c-4295-afaf-9953a6b45ffe admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:48.547765 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d219da5-032c-4295-afaf-9953a6b45ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:48.548044 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d219da5-032c-4295-afaf-9953a6b45ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:48.548864 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d219da5-032c-4295-afaf-9953a6b45ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:48.550900 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d219da5-032c-4295-afaf-9953a6b45ffe admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:48.563128 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1395/2786] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 23:12:48.571907 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c3e9a-e51a-4800-8337-9b8b188702d2 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:48.582232 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c3e9a-e51a-4800-8337-9b8b188702d2 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:48.586724 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e1fee7-5b65-49ef-beb7-38801c0dde88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:48.587642 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1fee7-5b65-49ef-beb7-38801c0dde88 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:48.587915 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1fee7-5b65-49ef-beb7-38801c0dde88 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:48.588182 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1fee7-5b65-49ef-beb7-38801c0dde88 admin admin] PATH_INFO: `/v3/projects/6440b9f0b0c847b7a5e0f659f1051c70/users/c8864e65287444d392a55514fb7cbfdc/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:48.602947 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e1fee7-5b65-49ef-beb7-38801c0dde88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6440b9f0b0c847b7a5e0f659f1051c70, user_id=c8864e65287444d392a55514fb7cbfdc, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:48.605886 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e1fee7-5b65-49ef-beb7-38801c0dde88 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:48.618403 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1392/2787] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:12:48 2022] PUT /identity/v3/projects/6440b9f0b0c847b7a5e0f659f1051c70/users/c8864e65287444d392a55514fb7cbfdc/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:12:48.629871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad9419-bf93-46f4-8e66-5b63dc82206e None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:48.630181 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad9419-bf93-46f4-8e66-5b63dc82206e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:48.630478 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad9419-bf93-46f4-8e66-5b63dc82206e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:48.657495 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-36ad9419-bf93-46f4-8e66-5b63dc82206e None None] MFA Rules not processed for user `c8864e65287444d392a55514fb7cbfdc`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:12:48.658735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-36ad9419-bf93-46f4-8e66-5b63dc82206e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:48.718681 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1396/2788] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:12:48 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:12:48.723915 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e1fee7-5b65-49ef-beb7-38801c0dde88 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:48.782087 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e1fee7-5b65-49ef-beb7-38801c0dde88 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:48.786748 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04278c30-8202-40cf-852f-75368d4c055d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:48.787389 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04278c30-8202-40cf-852f-75368d4c055d admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:48.787698 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04278c30-8202-40cf-852f-75368d4c055d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:48.787970 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04278c30-8202-40cf-852f-75368d4c055d admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:48.788647 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04278c30-8202-40cf-852f-75368d4c055d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:48.790567 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04278c30-8202-40cf-852f-75368d4c055d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:48.814435 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1393/2789] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:12:48 2022] POST /identity/v3/projects => generated 365 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:12:48.818166 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad9419-bf93-46f4-8e66-5b63dc82206e None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:48.828965 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad9419-bf93-46f4-8e66-5b63dc82206e None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:48.886190 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8daf38f-d1ec-4411-b909-beb1a3758d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:48.886784 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8daf38f-d1ec-4411-b909-beb1a3758d9d admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:48.887051 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8daf38f-d1ec-4411-b909-beb1a3758d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:48.887319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8daf38f-d1ec-4411-b909-beb1a3758d9d admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:48.887983 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8daf38f-d1ec-4411-b909-beb1a3758d9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:48.889883 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8daf38f-d1ec-4411-b909-beb1a3758d9d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:48.909307 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1397/2790] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:12:48 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:12:48.913501 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04278c30-8202-40cf-852f-75368d4c055d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:48.923554 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04278c30-8202-40cf-852f-75368d4c055d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:48.928145 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7c2bf-9716-415c-a92d-c601113f5ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:48.928790 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7c2bf-9716-415c-a92d-c601113f5ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:48.929072 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7c2bf-9716-415c-a92d-c601113f5ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:48.929348 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7c2bf-9716-415c-a92d-c601113f5ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:48.930179 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7c2bf-9716-415c-a92d-c601113f5ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:48.931761 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7c2bf-9716-415c-a92d-c601113f5ec3 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:48.942637 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1394/2791] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:12:48.946549 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8daf38f-d1ec-4411-b909-beb1a3758d9d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:48.957287 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8daf38f-d1ec-4411-b909-beb1a3758d9d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:48.962041 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e7d46-3479-4908-9182-207bd557cdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:48.962937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e7d46-3479-4908-9182-207bd557cdc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:48.963216 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e7d46-3479-4908-9182-207bd557cdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:48.963486 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e7d46-3479-4908-9182-207bd557cdc9 admin admin] PATH_INFO: `/v3/projects/047767f15c2f4550aedf033c06e2f5d7/users/f0168c4bb07e41eca4d45dc6cac95c95/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:48.977596 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e7d46-3479-4908-9182-207bd557cdc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=047767f15c2f4550aedf033c06e2f5d7, user_id=f0168c4bb07e41eca4d45dc6cac95c95, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:48.980544 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e7d46-3479-4908-9182-207bd557cdc9 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:48.992621 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1398/2792] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:12:48 2022] PUT /identity/v3/projects/047767f15c2f4550aedf033c06e2f5d7/users/f0168c4bb07e41eca4d45dc6cac95c95/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:12:48.996431 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7c2bf-9716-415c-a92d-c601113f5ec3 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:49.006684 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7c2bf-9716-415c-a92d-c601113f5ec3 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:49.059662 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5e797-9d9d-4091-91d5-18d1aa51212b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:49.060309 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5e797-9d9d-4091-91d5-18d1aa51212b admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:49.060594 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5e797-9d9d-4091-91d5-18d1aa51212b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:49.060869 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5e797-9d9d-4091-91d5-18d1aa51212b admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:49.061573 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5e797-9d9d-4091-91d5-18d1aa51212b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:49.063732 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5e797-9d9d-4091-91d5-18d1aa51212b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:49.073300 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1395/2793] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:12:49.077393 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e7d46-3479-4908-9182-207bd557cdc9 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:49.090495 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e7d46-3479-4908-9182-207bd557cdc9 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:49.095122 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb8257-c710-4063-9ce2-cc7cb43ba57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:49.096031 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb8257-c710-4063-9ce2-cc7cb43ba57a admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:49.096306 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb8257-c710-4063-9ce2-cc7cb43ba57a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:49.096577 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb8257-c710-4063-9ce2-cc7cb43ba57a admin admin] PATH_INFO: `/v3/projects/047767f15c2f4550aedf033c06e2f5d7/users/f0168c4bb07e41eca4d45dc6cac95c95/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:49.098685 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb8257-c710-4063-9ce2-cc7cb43ba57a admin admin] RBAC: Authorizing `identity:create_grant(project_id=047767f15c2f4550aedf033c06e2f5d7, user_id=f0168c4bb07e41eca4d45dc6cac95c95, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:49.101385 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb8257-c710-4063-9ce2-cc7cb43ba57a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:49.111805 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1399/2794] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:12:49 2022] PUT /identity/v3/projects/047767f15c2f4550aedf033c06e2f5d7/users/f0168c4bb07e41eca4d45dc6cac95c95/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:12:49.121742 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572946c-668c-41d6-9923-6513ca32a035 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:49.122028 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572946c-668c-41d6-9923-6513ca32a035 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:49.122332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572946c-668c-41d6-9923-6513ca32a035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:49.150767 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-7572946c-668c-41d6-9923-6513ca32a035 None None] MFA Rules not processed for user `f0168c4bb07e41eca4d45dc6cac95c95`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:12:49.151887 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7572946c-668c-41d6-9923-6513ca32a035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:49.217191 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1396/2795] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:12:49 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:12:49.228460 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb8257-c710-4063-9ce2-cc7cb43ba57a admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:49.230429 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-74cb8257-c710-4063-9ce2-cc7cb43ba57a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:49.285441 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb8257-c710-4063-9ce2-cc7cb43ba57a admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:49.289069 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708deea-f9c5-4561-aac9-b18d254dfad2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:49.289695 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708deea-f9c5-4561-aac9-b18d254dfad2 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:49.289973 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708deea-f9c5-4561-aac9-b18d254dfad2 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:49.290278 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708deea-f9c5-4561-aac9-b18d254dfad2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:49.299619 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708deea-f9c5-4561-aac9-b18d254dfad2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:49.301030 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708deea-f9c5-4561-aac9-b18d254dfad2 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:49.310139 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1400/2796] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:12:49 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:12:49.700644 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572946c-668c-41d6-9923-6513ca32a035 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:49.765903 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572946c-668c-41d6-9923-6513ca32a035 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:49.770424 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f835a1-0786-4697-ade9-d04023485a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:49.771280 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f835a1-0786-4697-ade9-d04023485a80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:49.771561 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f835a1-0786-4697-ade9-d04023485a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:49.771834 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f835a1-0786-4697-ade9-d04023485a80 admin admin] PATH_INFO: `/v3/users/c8864e65287444d392a55514fb7cbfdc` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:49.773007 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f835a1-0786-4697-ade9-d04023485a80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8864e65287444d392a55514fb7cbfdc)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:49.774947 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f835a1-0786-4697-ade9-d04023485a80 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:49.820443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-27f835a1-0786-4697-ade9-d04023485a80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8864e65287444d392a55514fb7cbfdc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:49.826945 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-27f835a1-0786-4697-ade9-d04023485a80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8864e65287444d392a55514fb7cbfdc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:49.827591 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-27f835a1-0786-4697-ade9-d04023485a80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8864e65287444d392a55514fb7cbfdc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:49.834404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-27f835a1-0786-4697-ade9-d04023485a80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8864e65287444d392a55514fb7cbfdc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:49.835020 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-27f835a1-0786-4697-ade9-d04023485a80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8864e65287444d392a55514fb7cbfdc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:49.852132 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1397/2797] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:12:49 2022] DELETE /identity/v3/users/c8864e65287444d392a55514fb7cbfdc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:12:49.857450 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5344366-7db8-4ba7-bb8e-aafb4f134c7e None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:49.857619 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5344366-7db8-4ba7-bb8e-aafb4f134c7e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:49.857901 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5344366-7db8-4ba7-bb8e-aafb4f134c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:49.875504 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-a5344366-7db8-4ba7-bb8e-aafb4f134c7e None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:12:49.876666 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a5344366-7db8-4ba7-bb8e-aafb4f134c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:49.933319 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1401/2798] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:12:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:12:49.943308 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f835a1-0786-4697-ade9-d04023485a80 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:49.944721 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-27f835a1-0786-4697-ade9-d04023485a80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:50.000103 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f835a1-0786-4697-ade9-d04023485a80 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:50.003771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45727f9-c7ab-4313-9b9a-9bf839d19083 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:50.004408 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45727f9-c7ab-4313-9b9a-9bf839d19083 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:50.004684 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45727f9-c7ab-4313-9b9a-9bf839d19083 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:50.004957 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45727f9-c7ab-4313-9b9a-9bf839d19083 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:50.013610 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45727f9-c7ab-4313-9b9a-9bf839d19083 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:50.015071 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45727f9-c7ab-4313-9b9a-9bf839d19083 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:50.024654 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1398/2799] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:12:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:12:50.280211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5344366-7db8-4ba7-bb8e-aafb4f134c7e None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:50.281641 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a5344366-7db8-4ba7-bb8e-aafb4f134c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:50.293133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5344366-7db8-4ba7-bb8e-aafb4f134c7e None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:50.298261 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f1978-ddfd-48ba-893c-d931f9af82f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:50.299100 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f1978-ddfd-48ba-893c-d931f9af82f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:50.299375 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f1978-ddfd-48ba-893c-d931f9af82f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:50.299667 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f1978-ddfd-48ba-893c-d931f9af82f3 admin admin] PATH_INFO: `/v3/projects/6440b9f0b0c847b7a5e0f659f1051c70` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:50.300837 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f1978-ddfd-48ba-893c-d931f9af82f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6440b9f0b0c847b7a5e0f659f1051c70)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:50.302767 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f1978-ddfd-48ba-893c-d931f9af82f3 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:50.347211 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2b3f1978-ddfd-48ba-893c-d931f9af82f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6440b9f0b0c847b7a5e0f659f1051c70', 'request_id': 'req-2b3f1978-ddfd-48ba-893c-d931f9af82f3'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:50.352825 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2b3f1978-ddfd-48ba-893c-d931f9af82f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6440b9f0b0c847b7a5e0f659f1051c70', 'request_id': 'req-2b3f1978-ddfd-48ba-893c-d931f9af82f3'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:50.359199 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2b3f1978-ddfd-48ba-893c-d931f9af82f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6440b9f0b0c847b7a5e0f659f1051c70', 'request_id': 'req-2b3f1978-ddfd-48ba-893c-d931f9af82f3'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:50.371719 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2b3f1978-ddfd-48ba-893c-d931f9af82f3 admin admin] The token cache is being invalidate because project 6440b9f0b0c847b7a5e0f659f1051c70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:12:50.372004 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2b3f1978-ddfd-48ba-893c-d931f9af82f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:50.372613 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2b3f1978-ddfd-48ba-893c-d931f9af82f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:50.374137 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1402/2800] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:12:50 2022] DELETE /identity/v3/projects/6440b9f0b0c847b7a5e0f659f1051c70 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:12:50.378909 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45727f9-c7ab-4313-9b9a-9bf839d19083 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:50.388108 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45727f9-c7ab-4313-9b9a-9bf839d19083 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:50.440521 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d848a-37a3-4247-809a-20ad914b49e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:50.441364 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d848a-37a3-4247-809a-20ad914b49e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:50.441633 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d848a-37a3-4247-809a-20ad914b49e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:50.441896 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d848a-37a3-4247-809a-20ad914b49e1 admin admin] PATH_INFO: `/v3/users/f0168c4bb07e41eca4d45dc6cac95c95` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:50.443088 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d848a-37a3-4247-809a-20ad914b49e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0168c4bb07e41eca4d45dc6cac95c95)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:50.444975 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d848a-37a3-4247-809a-20ad914b49e1 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:50.488984 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e97d848a-37a3-4247-809a-20ad914b49e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0168c4bb07e41eca4d45dc6cac95c95'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:50.495496 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e97d848a-37a3-4247-809a-20ad914b49e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0168c4bb07e41eca4d45dc6cac95c95'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:50.496148 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e97d848a-37a3-4247-809a-20ad914b49e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0168c4bb07e41eca4d45dc6cac95c95'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:50.502928 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e97d848a-37a3-4247-809a-20ad914b49e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0168c4bb07e41eca4d45dc6cac95c95'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:50.503642 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e97d848a-37a3-4247-809a-20ad914b49e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0168c4bb07e41eca4d45dc6cac95c95'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:50.519666 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1399/2801] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:12:50 2022] DELETE /identity/v3/users/f0168c4bb07e41eca4d45dc6cac95c95 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:12:50.780745 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f1978-ddfd-48ba-893c-d931f9af82f3 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:50.782987 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-2b3f1978-ddfd-48ba-893c-d931f9af82f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:50.795862 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f1978-ddfd-48ba-893c-d931f9af82f3 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:50.850888 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c6533-73e9-48fc-9b9f-f83e499ece0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:50.851739 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c6533-73e9-48fc-9b9f-f83e499ece0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:50.852020 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c6533-73e9-48fc-9b9f-f83e499ece0f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:50.852295 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c6533-73e9-48fc-9b9f-f83e499ece0f admin admin] PATH_INFO: `/v3/projects/047767f15c2f4550aedf033c06e2f5d7` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:50.853453 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26c6533-73e9-48fc-9b9f-f83e499ece0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=047767f15c2f4550aedf033c06e2f5d7)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:50.855713 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26c6533-73e9-48fc-9b9f-f83e499ece0f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:50.900926 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b26c6533-73e9-48fc-9b9f-f83e499ece0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '047767f15c2f4550aedf033c06e2f5d7', 'request_id': 'req-b26c6533-73e9-48fc-9b9f-f83e499ece0f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:50.906101 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b26c6533-73e9-48fc-9b9f-f83e499ece0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '047767f15c2f4550aedf033c06e2f5d7', 'request_id': 'req-b26c6533-73e9-48fc-9b9f-f83e499ece0f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:50.912580 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b26c6533-73e9-48fc-9b9f-f83e499ece0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '047767f15c2f4550aedf033c06e2f5d7', 'request_id': 'req-b26c6533-73e9-48fc-9b9f-f83e499ece0f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:50.923847 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b26c6533-73e9-48fc-9b9f-f83e499ece0f admin admin] The token cache is being invalidate because project 047767f15c2f4550aedf033c06e2f5d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:12:50.924130 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b26c6533-73e9-48fc-9b9f-f83e499ece0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:50.924773 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b26c6533-73e9-48fc-9b9f-f83e499ece0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:12:50.926298 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1403/2802] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:12:50 2022] DELETE /identity/v3/projects/047767f15c2f4550aedf033c06e2f5d7 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:12:50.931369 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b712bf-b424-4a0b-bfaf-6c6e05bcfaea None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:50.931783 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b712bf-b424-4a0b-bfaf-6c6e05bcfaea None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:50.931921 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b712bf-b424-4a0b-bfaf-6c6e05bcfaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:50.948959 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-c6b712bf-b424-4a0b-bfaf-6c6e05bcfaea None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:12:50.950169 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c6b712bf-b424-4a0b-bfaf-6c6e05bcfaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:51.007209 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1400/2803] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:12:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:12:51.014867 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7183a21-b404-4bcc-adfb-fa0ade107200 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:51.015154 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7183a21-b404-4bcc-adfb-fa0ade107200 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:51.016156 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7183a21-b404-4bcc-adfb-fa0ade107200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:51.034406 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e7183a21-b404-4bcc-adfb-fa0ade107200 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:12:51.035524 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e7183a21-b404-4bcc-adfb-fa0ade107200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:51.043300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1404/2804] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:12:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:12:51.047235 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b712bf-b424-4a0b-bfaf-6c6e05bcfaea None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:51.056427 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b712bf-b424-4a0b-bfaf-6c6e05bcfaea None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:51.060881 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fbe664-ce53-4255-a2b1-3b517124595e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:51.061513 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbe664-ce53-4255-a2b1-3b517124595e admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:51.061788 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbe664-ce53-4255-a2b1-3b517124595e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:51.062054 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbe664-ce53-4255-a2b1-3b517124595e admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:51.062794 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fbe664-ce53-4255-a2b1-3b517124595e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:12:51.063116 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fbe664-ce53-4255-a2b1-3b517124595e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:51.064665 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fbe664-ce53-4255-a2b1-3b517124595e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:51.075605 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1401/2805] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 23:12:51.080362 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7183a21-b404-4bcc-adfb-fa0ade107200 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:51.089232 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7183a21-b404-4bcc-adfb-fa0ade107200 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:51.093799 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3057f-3924-485e-a3fb-f2db8318e492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:51.095045 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3057f-3924-485e-a3fb-f2db8318e492 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:51.095332 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3057f-3924-485e-a3fb-f2db8318e492 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:51.095603 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3057f-3924-485e-a3fb-f2db8318e492 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:51.096288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3057f-3924-485e-a3fb-f2db8318e492 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:51.098210 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3057f-3924-485e-a3fb-f2db8318e492 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:51.123604 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1405/2806] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:12:51 2022] POST /identity/v3/projects => generated 363 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:12:51.128317 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fbe664-ce53-4255-a2b1-3b517124595e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:51.137832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fbe664-ce53-4255-a2b1-3b517124595e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:51.193793 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36aecc9-e99d-46c9-aad0-6de1e9b86bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:51.194467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36aecc9-e99d-46c9-aad0-6de1e9b86bff admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:51.194744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36aecc9-e99d-46c9-aad0-6de1e9b86bff admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:51.195012 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36aecc9-e99d-46c9-aad0-6de1e9b86bff admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:51.195699 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36aecc9-e99d-46c9-aad0-6de1e9b86bff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:51.197598 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36aecc9-e99d-46c9-aad0-6de1e9b86bff admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:51.217725 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1402/2807] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:12:51 2022] POST /identity/v3/users => generated 282 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:12:51.221746 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3057f-3924-485e-a3fb-f2db8318e492 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:51.232165 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3057f-3924-485e-a3fb-f2db8318e492 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:51.236748 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea23d2f-7fe6-441d-9998-97bd41b93746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:51.237397 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea23d2f-7fe6-441d-9998-97bd41b93746 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:51.237680 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea23d2f-7fe6-441d-9998-97bd41b93746 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:51.237956 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea23d2f-7fe6-441d-9998-97bd41b93746 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:51.238715 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea23d2f-7fe6-441d-9998-97bd41b93746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:51.240286 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea23d2f-7fe6-441d-9998-97bd41b93746 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:51.250812 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1406/2808] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:12:51.254510 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36aecc9-e99d-46c9-aad0-6de1e9b86bff admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:51.264176 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36aecc9-e99d-46c9-aad0-6de1e9b86bff admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:51.269323 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652180b0-9504-4edc-b5c9-4fd381a3c406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:51.269968 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652180b0-9504-4edc-b5c9-4fd381a3c406 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:51.270287 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652180b0-9504-4edc-b5c9-4fd381a3c406 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:51.270564 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652180b0-9504-4edc-b5c9-4fd381a3c406 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:51.271275 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652180b0-9504-4edc-b5c9-4fd381a3c406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:51.272847 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652180b0-9504-4edc-b5c9-4fd381a3c406 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:51.282258 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1403/2809] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:12:51.286352 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea23d2f-7fe6-441d-9998-97bd41b93746 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:51.295884 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea23d2f-7fe6-441d-9998-97bd41b93746 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:51.300502 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57017c07-579b-4b62-9229-ed988efce230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:51.301354 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57017c07-579b-4b62-9229-ed988efce230 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:51.301629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57017c07-579b-4b62-9229-ed988efce230 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:51.301902 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57017c07-579b-4b62-9229-ed988efce230 admin admin] PATH_INFO: `/v3/projects/c4de524d19074fb2a642b216de64d788/users/3e5ce417330643d19da25c0c86772fa6/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:51.316356 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57017c07-579b-4b62-9229-ed988efce230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4de524d19074fb2a642b216de64d788, user_id=3e5ce417330643d19da25c0c86772fa6, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:51.318996 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57017c07-579b-4b62-9229-ed988efce230 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:51.332002 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1407/2810] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:12:51 2022] PUT /identity/v3/projects/c4de524d19074fb2a642b216de64d788/users/3e5ce417330643d19da25c0c86772fa6/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:12:51.343033 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f604f60-669e-40c2-89a9-090788486e02 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:51.343322 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f604f60-669e-40c2-89a9-090788486e02 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:51.343601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f604f60-669e-40c2-89a9-090788486e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:51.370500 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-0f604f60-669e-40c2-89a9-090788486e02 None None] MFA Rules not processed for user `3e5ce417330643d19da25c0c86772fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:12:51.371701 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0f604f60-669e-40c2-89a9-090788486e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:51.431832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1404/2811] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:12:51 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:12:51.436252 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57017c07-579b-4b62-9229-ed988efce230 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:51.496590 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57017c07-579b-4b62-9229-ed988efce230 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:51.501785 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fece85-bd96-4d2b-b436-aa54bbc7041a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:51.502459 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fece85-bd96-4d2b-b436-aa54bbc7041a admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:51.502743 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fece85-bd96-4d2b-b436-aa54bbc7041a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:51.503015 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fece85-bd96-4d2b-b436-aa54bbc7041a admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:51.503690 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fece85-bd96-4d2b-b436-aa54bbc7041a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:51.505558 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fece85-bd96-4d2b-b436-aa54bbc7041a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:51.529604 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1408/2812] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:12:51 2022] POST /identity/v3/projects => generated 365 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:12:51.533637 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f604f60-669e-40c2-89a9-090788486e02 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:51.544154 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f604f60-669e-40c2-89a9-090788486e02 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:51.597813 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d2ad71-c84d-44ea-9ed5-689f24895c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:51.598581 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2ad71-c84d-44ea-9ed5-689f24895c51 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:51.598866 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2ad71-c84d-44ea-9ed5-689f24895c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:51.599143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2ad71-c84d-44ea-9ed5-689f24895c51 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:51.599837 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d2ad71-c84d-44ea-9ed5-689f24895c51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:51.601744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d2ad71-c84d-44ea-9ed5-689f24895c51 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:51.621222 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1405/2813] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:12:51 2022] POST /identity/v3/users => generated 289 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:12:51.627322 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fece85-bd96-4d2b-b436-aa54bbc7041a admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:51.637441 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fece85-bd96-4d2b-b436-aa54bbc7041a admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:51.642060 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182862bb-11ee-4ba8-a688-1fdec6d80b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:51.642738 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182862bb-11ee-4ba8-a688-1fdec6d80b2e admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:51.643021 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182862bb-11ee-4ba8-a688-1fdec6d80b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:51.643296 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182862bb-11ee-4ba8-a688-1fdec6d80b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:51.644029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182862bb-11ee-4ba8-a688-1fdec6d80b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:51.645612 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182862bb-11ee-4ba8-a688-1fdec6d80b2e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:51.657248 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1409/2814] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:12:51.661466 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d2ad71-c84d-44ea-9ed5-689f24895c51 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:51.671257 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d2ad71-c84d-44ea-9ed5-689f24895c51 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:51.676322 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a36e3e-70b3-4618-8a1e-bcab4be0f344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:51.677186 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a36e3e-70b3-4618-8a1e-bcab4be0f344 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:51.677462 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a36e3e-70b3-4618-8a1e-bcab4be0f344 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:51.677730 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a36e3e-70b3-4618-8a1e-bcab4be0f344 admin admin] PATH_INFO: `/v3/projects/f6c0884f22a0484a97ef56a5a469c288/users/cb35908d89cf4876bed7fcd8e3aef672/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:51.693436 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a36e3e-70b3-4618-8a1e-bcab4be0f344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6c0884f22a0484a97ef56a5a469c288, user_id=cb35908d89cf4876bed7fcd8e3aef672, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:51.696996 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a36e3e-70b3-4618-8a1e-bcab4be0f344 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:51.711066 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1406/2815] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:12:51 2022] PUT /identity/v3/projects/f6c0884f22a0484a97ef56a5a469c288/users/cb35908d89cf4876bed7fcd8e3aef672/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:12:51.714969 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182862bb-11ee-4ba8-a688-1fdec6d80b2e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:51.724769 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182862bb-11ee-4ba8-a688-1fdec6d80b2e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:51.779289 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273de939-29c7-45ed-b929-aaf86f1a2ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:51.779941 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273de939-29c7-45ed-b929-aaf86f1a2ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:51.780215 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273de939-29c7-45ed-b929-aaf86f1a2ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:51.780486 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273de939-29c7-45ed-b929-aaf86f1a2ac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:51.781190 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273de939-29c7-45ed-b929-aaf86f1a2ac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:51.782792 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273de939-29c7-45ed-b929-aaf86f1a2ac3 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:51.792066 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1410/2816] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:12:51.796377 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a36e3e-70b3-4618-8a1e-bcab4be0f344 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:51.807643 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a36e3e-70b3-4618-8a1e-bcab4be0f344 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:51.812356 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7097436-f011-4f62-a18c-13efb9787d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:51.813811 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7097436-f011-4f62-a18c-13efb9787d10 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:51.814087 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7097436-f011-4f62-a18c-13efb9787d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:51.814380 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7097436-f011-4f62-a18c-13efb9787d10 admin admin] PATH_INFO: `/v3/projects/f6c0884f22a0484a97ef56a5a469c288/users/cb35908d89cf4876bed7fcd8e3aef672/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:51.816404 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7097436-f011-4f62-a18c-13efb9787d10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6c0884f22a0484a97ef56a5a469c288, user_id=cb35908d89cf4876bed7fcd8e3aef672, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:51.819186 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7097436-f011-4f62-a18c-13efb9787d10 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:51.829103 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1407/2817] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:12:51 2022] PUT /identity/v3/projects/f6c0884f22a0484a97ef56a5a469c288/users/cb35908d89cf4876bed7fcd8e3aef672/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:12:51.839317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff014a2-2a89-49d7-99a0-5073f33a2ca1 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:51.839601 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff014a2-2a89-49d7-99a0-5073f33a2ca1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:51.839877 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff014a2-2a89-49d7-99a0-5073f33a2ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:51.867482 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-dff014a2-2a89-49d7-99a0-5073f33a2ca1 None None] MFA Rules not processed for user `cb35908d89cf4876bed7fcd8e3aef672`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:12:51.868687 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-dff014a2-2a89-49d7-99a0-5073f33a2ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:51.932231 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1411/2818] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:12:51 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:12:51.943217 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7097436-f011-4f62-a18c-13efb9787d10 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:51.944913 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a7097436-f011-4f62-a18c-13efb9787d10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:12:51.998999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7097436-f011-4f62-a18c-13efb9787d10 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:52.003577 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6421a03-1eb6-44bd-9981-bafacc28112d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:52.004197 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6421a03-1eb6-44bd-9981-bafacc28112d service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:52.004473 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6421a03-1eb6-44bd-9981-bafacc28112d service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:52.004744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6421a03-1eb6-44bd-9981-bafacc28112d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:52.013217 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6421a03-1eb6-44bd-9981-bafacc28112d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:52.014658 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6421a03-1eb6-44bd-9981-bafacc28112d service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:52.024389 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1408/2819] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:12:51 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:12:52.095317 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff014a2-2a89-49d7-99a0-5073f33a2ca1 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:12:52.104667 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff014a2-2a89-49d7-99a0-5073f33a2ca1 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:12:52.108577 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb000866-2934-4a70-8fc9-2d5562210a50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:12:52.109202 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb000866-2934-4a70-8fc9-2d5562210a50 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:12:52.110097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb000866-2934-4a70-8fc9-2d5562210a50 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:12:52.110404 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb000866-2934-4a70-8fc9-2d5562210a50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:12:52.165555 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb000866-2934-4a70-8fc9-2d5562210a50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:12:52.167023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb000866-2934-4a70-8fc9-2d5562210a50 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:12:52.176364 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1412/2820] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:12:52 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:13:01.177161 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6421a03-1eb6-44bd-9981-bafacc28112d service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:13:01.242758 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6421a03-1eb6-44bd-9981-bafacc28112d service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:13:01.249045 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca593f4-83bc-46a7-9ab2-2c8902a60653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:13:01.249992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca593f4-83bc-46a7-9ab2-2c8902a60653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:13:01.250878 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca593f4-83bc-46a7-9ab2-2c8902a60653 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:13:01.251184 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca593f4-83bc-46a7-9ab2-2c8902a60653 admin admin] PATH_INFO: `/v3/users/3e5ce417330643d19da25c0c86772fa6` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:13:01.252423 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca593f4-83bc-46a7-9ab2-2c8902a60653 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e5ce417330643d19da25c0c86772fa6)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:13:01.255489 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca593f4-83bc-46a7-9ab2-2c8902a60653 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:13:01.300647 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8ca593f4-83bc-46a7-9ab2-2c8902a60653 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e5ce417330643d19da25c0c86772fa6'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:01.307138 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8ca593f4-83bc-46a7-9ab2-2c8902a60653 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e5ce417330643d19da25c0c86772fa6'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:01.307792 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8ca593f4-83bc-46a7-9ab2-2c8902a60653 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e5ce417330643d19da25c0c86772fa6'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:01.315529 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8ca593f4-83bc-46a7-9ab2-2c8902a60653 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e5ce417330643d19da25c0c86772fa6'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:01.316282 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8ca593f4-83bc-46a7-9ab2-2c8902a60653 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e5ce417330643d19da25c0c86772fa6'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:01.332438 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1409/2821] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:13:01 2022] DELETE /identity/v3/users/3e5ce417330643d19da25c0c86772fa6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:13:01.337094 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ab0c7-24c2-443e-8731-b8b6d3665393 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:13:01.337580 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ab0c7-24c2-443e-8731-b8b6d3665393 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:13:01.337695 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ab0c7-24c2-443e-8731-b8b6d3665393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:13:01.354006 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-420ab0c7-24c2-443e-8731-b8b6d3665393 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:13:01.355492 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-420ab0c7-24c2-443e-8731-b8b6d3665393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:13:01.411338 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1413/2822] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:13:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:13:01.420940 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca593f4-83bc-46a7-9ab2-2c8902a60653 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:13:01.422953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8ca593f4-83bc-46a7-9ab2-2c8902a60653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:13:01.477695 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca593f4-83bc-46a7-9ab2-2c8902a60653 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:13:01.481629 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd88ec-f92d-4857-b9a9-a8ca18f0a1ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:13:01.482290 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd88ec-f92d-4857-b9a9-a8ca18f0a1ad service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:13:01.482596 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd88ec-f92d-4857-b9a9-a8ca18f0a1ad service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:13:01.482873 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd88ec-f92d-4857-b9a9-a8ca18f0a1ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:13:01.492400 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd88ec-f92d-4857-b9a9-a8ca18f0a1ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:13:01.493831 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd88ec-f92d-4857-b9a9-a8ca18f0a1ad service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:13:01.503704 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1410/2823] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:13:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:13:01.760026 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ab0c7-24c2-443e-8731-b8b6d3665393 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:13:01.761350 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-420ab0c7-24c2-443e-8731-b8b6d3665393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:13:01.774959 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ab0c7-24c2-443e-8731-b8b6d3665393 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:13:01.779624 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432592b-6734-49b1-b105-a710ac150765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:13:01.780945 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432592b-6734-49b1-b105-a710ac150765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:13:01.781913 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432592b-6734-49b1-b105-a710ac150765 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:13:01.782234 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432592b-6734-49b1-b105-a710ac150765 admin admin] PATH_INFO: `/v3/projects/c4de524d19074fb2a642b216de64d788` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:13:01.783427 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432592b-6734-49b1-b105-a710ac150765 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4de524d19074fb2a642b216de64d788)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:13:01.785324 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432592b-6734-49b1-b105-a710ac150765 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:13:01.830099 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e432592b-6734-49b1-b105-a710ac150765 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4de524d19074fb2a642b216de64d788', 'request_id': 'req-e432592b-6734-49b1-b105-a710ac150765'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:01.835368 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e432592b-6734-49b1-b105-a710ac150765 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4de524d19074fb2a642b216de64d788', 'request_id': 'req-e432592b-6734-49b1-b105-a710ac150765'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:01.841299 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e432592b-6734-49b1-b105-a710ac150765 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4de524d19074fb2a642b216de64d788', 'request_id': 'req-e432592b-6734-49b1-b105-a710ac150765'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:01.853509 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e432592b-6734-49b1-b105-a710ac150765 admin admin] The token cache is being invalidate because project c4de524d19074fb2a642b216de64d788 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:13:01.853799 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e432592b-6734-49b1-b105-a710ac150765 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:01.854429 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e432592b-6734-49b1-b105-a710ac150765 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:01.856205 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1414/2824] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:13:01 2022] DELETE /identity/v3/projects/c4de524d19074fb2a642b216de64d788 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:13:01.861230 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd88ec-f92d-4857-b9a9-a8ca18f0a1ad service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:13:01.870761 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd88ec-f92d-4857-b9a9-a8ca18f0a1ad service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:13:01.923788 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf665821-bda4-47ad-80bd-03dc72589216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:13:01.924640 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf665821-bda4-47ad-80bd-03dc72589216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:13:01.924919 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf665821-bda4-47ad-80bd-03dc72589216 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:13:01.925193 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf665821-bda4-47ad-80bd-03dc72589216 admin admin] PATH_INFO: `/v3/users/cb35908d89cf4876bed7fcd8e3aef672` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:13:01.926387 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf665821-bda4-47ad-80bd-03dc72589216 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb35908d89cf4876bed7fcd8e3aef672)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:13:01.928296 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf665821-bda4-47ad-80bd-03dc72589216 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:13:01.972923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cf665821-bda4-47ad-80bd-03dc72589216 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb35908d89cf4876bed7fcd8e3aef672'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:01.979467 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cf665821-bda4-47ad-80bd-03dc72589216 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb35908d89cf4876bed7fcd8e3aef672'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:01.980361 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cf665821-bda4-47ad-80bd-03dc72589216 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb35908d89cf4876bed7fcd8e3aef672'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:01.987320 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cf665821-bda4-47ad-80bd-03dc72589216 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb35908d89cf4876bed7fcd8e3aef672'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:01.988023 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cf665821-bda4-47ad-80bd-03dc72589216 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb35908d89cf4876bed7fcd8e3aef672'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:02.004199 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1411/2825] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:13:01 2022] DELETE /identity/v3/users/cb35908d89cf4876bed7fcd8e3aef672 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:13:02.261256 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432592b-6734-49b1-b105-a710ac150765 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:13:02.262813 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e432592b-6734-49b1-b105-a710ac150765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:13:02.276434 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432592b-6734-49b1-b105-a710ac150765 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:13:02.332219 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe33e9-28b3-4bcc-bab4-e97cb5541e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:13:02.333071 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe33e9-28b3-4bcc-bab4-e97cb5541e3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:13:02.333352 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe33e9-28b3-4bcc-bab4-e97cb5541e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:13:02.333629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe33e9-28b3-4bcc-bab4-e97cb5541e3a admin admin] PATH_INFO: `/v3/projects/f6c0884f22a0484a97ef56a5a469c288` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:13:02.334824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fe33e9-28b3-4bcc-bab4-e97cb5541e3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6c0884f22a0484a97ef56a5a469c288)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:13:02.336728 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fe33e9-28b3-4bcc-bab4-e97cb5541e3a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:13:02.382148 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-43fe33e9-28b3-4bcc-bab4-e97cb5541e3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6c0884f22a0484a97ef56a5a469c288', 'request_id': 'req-43fe33e9-28b3-4bcc-bab4-e97cb5541e3a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:02.387286 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-43fe33e9-28b3-4bcc-bab4-e97cb5541e3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6c0884f22a0484a97ef56a5a469c288', 'request_id': 'req-43fe33e9-28b3-4bcc-bab4-e97cb5541e3a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:02.393888 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-43fe33e9-28b3-4bcc-bab4-e97cb5541e3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6c0884f22a0484a97ef56a5a469c288', 'request_id': 'req-43fe33e9-28b3-4bcc-bab4-e97cb5541e3a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:02.405671 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-43fe33e9-28b3-4bcc-bab4-e97cb5541e3a admin admin] The token cache is being invalidate because project f6c0884f22a0484a97ef56a5a469c288 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:13:02.405952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-43fe33e9-28b3-4bcc-bab4-e97cb5541e3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:02.406601 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-43fe33e9-28b3-4bcc-bab4-e97cb5541e3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:02.408115 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1415/2826] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:13:02 2022] DELETE /identity/v3/projects/f6c0884f22a0484a97ef56a5a469c288 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:13:02.412971 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afab17-90d7-4f5d-8f47-a77f54212cd2 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:13:02.413443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afab17-90d7-4f5d-8f47-a77f54212cd2 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:13:02.413548 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afab17-90d7-4f5d-8f47-a77f54212cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:13:02.430528 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-40afab17-90d7-4f5d-8f47-a77f54212cd2 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:13:02.431744 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-40afab17-90d7-4f5d-8f47-a77f54212cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:13:02.488463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1412/2827] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:13:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:13:02.497367 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9a154-d37d-4f66-a20e-5d8d2bb15e82 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:13:02.497645 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9a154-d37d-4f66-a20e-5d8d2bb15e82 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:13:02.497915 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9a154-d37d-4f66-a20e-5d8d2bb15e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:13:02.516553 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-94f9a154-d37d-4f66-a20e-5d8d2bb15e82 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:13:02.517727 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-94f9a154-d37d-4f66-a20e-5d8d2bb15e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:13:02.525401 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1416/2828] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:13:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:13:02.529389 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afab17-90d7-4f5d-8f47-a77f54212cd2 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:13:02.539036 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afab17-90d7-4f5d-8f47-a77f54212cd2 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:13:02.543601 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff76c5c-d42c-48a6-ae08-60ce351bbe7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:13:02.544243 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff76c5c-d42c-48a6-ae08-60ce351bbe7f admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:13:02.544523 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff76c5c-d42c-48a6-ae08-60ce351bbe7f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:13:02.544802 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff76c5c-d42c-48a6-ae08-60ce351bbe7f admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:13:02.545512 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff76c5c-d42c-48a6-ae08-60ce351bbe7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:13:02.545830 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff76c5c-d42c-48a6-ae08-60ce351bbe7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:13:02.547430 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff76c5c-d42c-48a6-ae08-60ce351bbe7f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:13:02.559027 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1413/2829] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:13:02.563678 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9a154-d37d-4f66-a20e-5d8d2bb15e82 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:13:02.572560 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9a154-d37d-4f66-a20e-5d8d2bb15e82 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:13:02.577167 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449186ce-f2b8-4f9d-acb4-536411dc8f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:13:02.577794 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449186ce-f2b8-4f9d-acb4-536411dc8f0c admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:13:02.578074 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449186ce-f2b8-4f9d-acb4-536411dc8f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:13:02.578976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449186ce-f2b8-4f9d-acb4-536411dc8f0c admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:13:02.579661 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449186ce-f2b8-4f9d-acb4-536411dc8f0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:13:02.581523 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449186ce-f2b8-4f9d-acb4-536411dc8f0c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:13:02.612215 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1417/2830] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:13:02 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:13:02.617517 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff76c5c-d42c-48a6-ae08-60ce351bbe7f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:13:02.628456 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff76c5c-d42c-48a6-ae08-60ce351bbe7f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:13:02.682331 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022cfbf-931a-4241-8c3e-73bfb698f2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:13:02.682922 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022cfbf-931a-4241-8c3e-73bfb698f2dc admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:13:02.683202 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022cfbf-931a-4241-8c3e-73bfb698f2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:13:02.683470 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022cfbf-931a-4241-8c3e-73bfb698f2dc admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:13:02.684141 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022cfbf-931a-4241-8c3e-73bfb698f2dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:13:02.686020 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022cfbf-931a-4241-8c3e-73bfb698f2dc admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:13:02.705908 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1414/2831] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:13:02 2022] POST /identity/v3/users => generated 287 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:13:02.710001 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449186ce-f2b8-4f9d-acb4-536411dc8f0c admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:13:02.720110 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449186ce-f2b8-4f9d-acb4-536411dc8f0c admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:13:02.724620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26061afb-72fd-4a23-95e5-7f62d357f44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:13:02.725264 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26061afb-72fd-4a23-95e5-7f62d357f44e admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:13:02.725543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26061afb-72fd-4a23-95e5-7f62d357f44e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:13:02.725817 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26061afb-72fd-4a23-95e5-7f62d357f44e admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:13:02.726559 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26061afb-72fd-4a23-95e5-7f62d357f44e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:13:02.728131 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26061afb-72fd-4a23-95e5-7f62d357f44e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:13:02.738937 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1418/2832] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:13:02.742668 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022cfbf-931a-4241-8c3e-73bfb698f2dc admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:13:02.752931 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022cfbf-931a-4241-8c3e-73bfb698f2dc admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:13:02.757516 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54428e18-1ac5-44fa-96f0-f7975062386f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:13:02.758185 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54428e18-1ac5-44fa-96f0-f7975062386f admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:13:02.758470 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54428e18-1ac5-44fa-96f0-f7975062386f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:13:02.758749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54428e18-1ac5-44fa-96f0-f7975062386f admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:13:02.759462 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54428e18-1ac5-44fa-96f0-f7975062386f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:13:02.761037 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54428e18-1ac5-44fa-96f0-f7975062386f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:13:02.770593 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1415/2833] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:13:02.774442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26061afb-72fd-4a23-95e5-7f62d357f44e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:13:02.784247 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26061afb-72fd-4a23-95e5-7f62d357f44e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:13:02.788773 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4b4da-3d05-4a4c-9582-962e440bca21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:13:02.789629 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4b4da-3d05-4a4c-9582-962e440bca21 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:13:02.789914 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4b4da-3d05-4a4c-9582-962e440bca21 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:13:02.790224 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4b4da-3d05-4a4c-9582-962e440bca21 admin admin] PATH_INFO: `/v3/projects/62ea39b51858417ebc3fdb8b624caa6e/users/e4e12e04384d4af0859b553936d6dbe1/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:13:02.804580 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4b4da-3d05-4a4c-9582-962e440bca21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62ea39b51858417ebc3fdb8b624caa6e, user_id=e4e12e04384d4af0859b553936d6dbe1, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:13:02.807253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4b4da-3d05-4a4c-9582-962e440bca21 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:13:02.820098 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1419/2834] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:13:02 2022] PUT /identity/v3/projects/62ea39b51858417ebc3fdb8b624caa6e/users/e4e12e04384d4af0859b553936d6dbe1/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:13:02.880708 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb52bf0-62e7-40d2-82e9-3a33f57e3f74 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:13:02.880999 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb52bf0-62e7-40d2-82e9-3a33f57e3f74 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:13:02.881298 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb52bf0-62e7-40d2-82e9-3a33f57e3f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:13:02.907143 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-9cb52bf0-62e7-40d2-82e9-3a33f57e3f74 None None] MFA Rules not processed for user `e4e12e04384d4af0859b553936d6dbe1`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:13:02.908498 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9cb52bf0-62e7-40d2-82e9-3a33f57e3f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:13:02.970975 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1416/2835] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:13:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:13:02.982311 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4b4da-3d05-4a4c-9582-962e440bca21 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:13:02.983967 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-12c4b4da-3d05-4a4c-9582-962e440bca21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:13:03.038876 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4b4da-3d05-4a4c-9582-962e440bca21 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:13:03.042602 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ca018-a3a9-4aab-98be-ca5c89aab548 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe668a2cee040c7a56d1f8685744cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:13:03.043263 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ca018-a3a9-4aab-98be-ca5c89aab548 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:13:03.043545 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ca018-a3a9-4aab-98be-ca5c89aab548 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:13:03.043824 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ca018-a3a9-4aab-98be-ca5c89aab548 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:13:03.052574 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ca018-a3a9-4aab-98be-ca5c89aab548 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:13:03.053987 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ca018-a3a9-4aab-98be-ca5c89aab548 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:13:03.062448 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1420/2836] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:13:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:13:12.816552 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb52bf0-62e7-40d2-82e9-3a33f57e3f74 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:13:12.877473 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb52bf0-62e7-40d2-82e9-3a33f57e3f74 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:13:12.882216 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cab65d-4279-47d6-84be-b945ff392324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:13:12.883024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cab65d-4279-47d6-84be-b945ff392324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:13:12.883304 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cab65d-4279-47d6-84be-b945ff392324 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:13:12.883578 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cab65d-4279-47d6-84be-b945ff392324 admin admin] PATH_INFO: `/v3/users/e4e12e04384d4af0859b553936d6dbe1` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:13:12.884756 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cab65d-4279-47d6-84be-b945ff392324 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4e12e04384d4af0859b553936d6dbe1)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:13:12.886703 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cab65d-4279-47d6-84be-b945ff392324 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:13:12.930823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-20cab65d-4279-47d6-84be-b945ff392324 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4e12e04384d4af0859b553936d6dbe1'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:12.937390 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-20cab65d-4279-47d6-84be-b945ff392324 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4e12e04384d4af0859b553936d6dbe1'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:12.938028 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-20cab65d-4279-47d6-84be-b945ff392324 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4e12e04384d4af0859b553936d6dbe1'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:12.945443 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-20cab65d-4279-47d6-84be-b945ff392324 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4e12e04384d4af0859b553936d6dbe1'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:12.946048 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-20cab65d-4279-47d6-84be-b945ff392324 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4e12e04384d4af0859b553936d6dbe1'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:12.963070 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1417/2837] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:13:12 2022] DELETE /identity/v3/users/e4e12e04384d4af0859b553936d6dbe1 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:13:12.967537 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13664e15-5af9-4376-b0d6-e4bafcf5feeb None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:13:12.968011 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13664e15-5af9-4376-b0d6-e4bafcf5feeb None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:13:12.968111 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13664e15-5af9-4376-b0d6-e4bafcf5feeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:13:12.985152 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-13664e15-5af9-4376-b0d6-e4bafcf5feeb None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:13:12.986462 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-13664e15-5af9-4376-b0d6-e4bafcf5feeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:13:13.041615 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1421/2838] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:13:13.051328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cab65d-4279-47d6-84be-b945ff392324 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:13:13.052700 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-20cab65d-4279-47d6-84be-b945ff392324 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:13:13.109418 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cab65d-4279-47d6-84be-b945ff392324 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:13:13.113778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a578a-5341-449b-ba45-d95d563d5166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:13:13.114669 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a578a-5341-449b-ba45-d95d563d5166 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:13:13.114957 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a578a-5341-449b-ba45-d95d563d5166 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:13:13.115231 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a578a-5341-449b-ba45-d95d563d5166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:13:13.124028 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a578a-5341-449b-ba45-d95d563d5166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:13:13.125442 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a578a-5341-449b-ba45-d95d563d5166 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:13:13.134720 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1418/2839] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:13:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 23:13:13.397609 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13664e15-5af9-4376-b0d6-e4bafcf5feeb None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:13:13.399075 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-13664e15-5af9-4376-b0d6-e4bafcf5feeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:13:13.411769 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13664e15-5af9-4376-b0d6-e4bafcf5feeb None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:13:13.416573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60339ef3-ea8a-4cbc-ac3f-99aea30cd0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:13:13.417518 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60339ef3-ea8a-4cbc-ac3f-99aea30cd0c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:13:13.417802 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60339ef3-ea8a-4cbc-ac3f-99aea30cd0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:13:13.418076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60339ef3-ea8a-4cbc-ac3f-99aea30cd0c3 admin admin] PATH_INFO: `/v3/projects/62ea39b51858417ebc3fdb8b624caa6e` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:13:13.419643 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60339ef3-ea8a-4cbc-ac3f-99aea30cd0c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62ea39b51858417ebc3fdb8b624caa6e)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:13:13.421560 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60339ef3-ea8a-4cbc-ac3f-99aea30cd0c3 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:13:13.467266 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-60339ef3-ea8a-4cbc-ac3f-99aea30cd0c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62ea39b51858417ebc3fdb8b624caa6e', 'request_id': 'req-60339ef3-ea8a-4cbc-ac3f-99aea30cd0c3'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:13.473031 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-60339ef3-ea8a-4cbc-ac3f-99aea30cd0c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62ea39b51858417ebc3fdb8b624caa6e', 'request_id': 'req-60339ef3-ea8a-4cbc-ac3f-99aea30cd0c3'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:13.479372 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-60339ef3-ea8a-4cbc-ac3f-99aea30cd0c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62ea39b51858417ebc3fdb8b624caa6e', 'request_id': 'req-60339ef3-ea8a-4cbc-ac3f-99aea30cd0c3'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:13.491589 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-60339ef3-ea8a-4cbc-ac3f-99aea30cd0c3 admin admin] The token cache is being invalidate because project 62ea39b51858417ebc3fdb8b624caa6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:13:13.491879 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-60339ef3-ea8a-4cbc-ac3f-99aea30cd0c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:13.492499 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-60339ef3-ea8a-4cbc-ac3f-99aea30cd0c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:13:13.494023 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1422/2840] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:13:13 2022] DELETE /identity/v3/projects/62ea39b51858417ebc3fdb8b624caa6e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:13:29.084450 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a578a-5341-449b-ba45-d95d563d5166 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:13:29.085970 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-119a578a-5341-449b-ba45-d95d563d5166 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:13:29.097508 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a578a-5341-449b-ba45-d95d563d5166 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:13:29.144825 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4459ce-4994-41d4-9e77-1a0af342e6d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:13:29.145463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4459ce-4994-41d4-9e77-1a0af342e6d1 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:13:29.145738 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4459ce-4994-41d4-9e77-1a0af342e6d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:13:29.146009 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4459ce-4994-41d4-9e77-1a0af342e6d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:13:29.147757 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-fc4459ce-4994-41d4-9e77-1a0af342e6d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:13:29.156623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4459ce-4994-41d4-9e77-1a0af342e6d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:13:29.158024 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4459ce-4994-41d4-9e77-1a0af342e6d1 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:13:29.166357 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1419/2841] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:13:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:13:59.284585 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fe856-15f6-4ce0-bfec-392fb4ef1829 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:13:59.285175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fe856-15f6-4ce0-bfec-392fb4ef1829 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:13:59.285175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fe856-15f6-4ce0-bfec-392fb4ef1829 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:13:59.286563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1423/2842] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 9 23:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 23:13:59.292876 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb352c46-7a78-4cf6-ad51-7edf14ce64dc None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:13:59.293340 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb352c46-7a78-4cf6-ad51-7edf14ce64dc None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:13:59.293441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb352c46-7a78-4cf6-ad51-7edf14ce64dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:13:59.331915 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-cb352c46-7a78-4cf6-ad51-7edf14ce64dc None None] MFA Rules not processed for user `96fd3f6dd9e74ba7a4ad159e43e8ae96`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:13:59.333120 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-cb352c46-7a78-4cf6-ad51-7edf14ce64dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:13:59.339799 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1420/2843] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 9 23:13:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:13:59.352443 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fe856-15f6-4ce0-bfec-392fb4ef1829 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:13:59.354921 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f69fe856-15f6-4ce0-bfec-392fb4ef1829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:13:59.379948 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fe856-15f6-4ce0-bfec-392fb4ef1829 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:13:59.383942 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d90b6-8ad7-442a-a802-92f3c07413df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:13:59.384620 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d90b6-8ad7-442a-a802-92f3c07413df service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:13:59.384895 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d90b6-8ad7-442a-a802-92f3c07413df service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:13:59.385164 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d90b6-8ad7-442a-a802-92f3c07413df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:13:59.392521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d90b6-8ad7-442a-a802-92f3c07413df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:13:59.394009 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d90b6-8ad7-442a-a802-92f3c07413df service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:13:59.403963 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1424/2844] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:13:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:14:00.494709 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815ac16-3539-4817-99ec-717dbfc87728 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:14:00.495201 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815ac16-3539-4817-99ec-717dbfc87728 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:14:00.495297 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815ac16-3539-4817-99ec-717dbfc87728 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:14:00.496631 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1421/2845] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 9 23:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 23:14:00.502236 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa2a80-4d09-490f-872b-53e89caeab5b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:14:00.502702 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa2a80-4d09-490f-872b-53e89caeab5b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:14:00.502810 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa2a80-4d09-490f-872b-53e89caeab5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:14:00.521852 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-26fa2a80-4d09-490f-872b-53e89caeab5b None None] MFA Rules not processed for user `02fcd7cc86134ce488c7dd75dfb174b8`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:14:00.523097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-26fa2a80-4d09-490f-872b-53e89caeab5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:14:00.587813 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1425/2846] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 9 23:14:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 23:14:00.600359 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815ac16-3539-4817-99ec-717dbfc87728 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:14:00.611274 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815ac16-3539-4817-99ec-717dbfc87728 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:14:00.614876 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585490b-4df2-4dc3-859d-b3434c3171e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:14:00.615514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585490b-4df2-4dc3-859d-b3434c3171e0 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:14:00.615794 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585490b-4df2-4dc3-859d-b3434c3171e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:14:00.616063 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585490b-4df2-4dc3-859d-b3434c3171e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:14:00.626068 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585490b-4df2-4dc3-859d-b3434c3171e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:14:00.627605 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585490b-4df2-4dc3-859d-b3434c3171e0 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:14:00.638780 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1422/2847] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:14:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 23:14:00.702467 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa2a80-4d09-490f-872b-53e89caeab5b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:14:00.703843 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-26fa2a80-4d09-490f-872b-53e89caeab5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:14:00.777503 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa2a80-4d09-490f-872b-53e89caeab5b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:14:00.782547 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb7649-0b98-4b05-b2ae-5b2434c193d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf9bc2fa959d4706b9f9e87509829059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:14:00.783192 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb7649-0b98-4b05-b2ae-5b2434c193d6 service placement] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:14:00.783477 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb7649-0b98-4b05-b2ae-5b2434c193d6 service placement] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:14:00.783752 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb7649-0b98-4b05-b2ae-5b2434c193d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:14:00.785493 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-cecb7649-0b98-4b05-b2ae-5b2434c193d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:14:00.797007 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb7649-0b98-4b05-b2ae-5b2434c193d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:14:00.798597 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb7649-0b98-4b05-b2ae-5b2434c193d6 service placement] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:14:00.808797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1426/2848] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:14:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:14:03.196676 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585490b-4df2-4dc3-859d-b3434c3171e0 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:14:03.198271 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f585490b-4df2-4dc3-859d-b3434c3171e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:14:03.213728 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585490b-4df2-4dc3-859d-b3434c3171e0 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:14:03.219291 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87461d65-5012-46de-a7d9-48cd26ec3029 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:14:03.219926 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87461d65-5012-46de-a7d9-48cd26ec3029 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:14:03.220209 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87461d65-5012-46de-a7d9-48cd26ec3029 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:14:03.220481 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87461d65-5012-46de-a7d9-48cd26ec3029 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:14:03.222325 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-87461d65-5012-46de-a7d9-48cd26ec3029 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:14:03.235432 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87461d65-5012-46de-a7d9-48cd26ec3029 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:14:03.237032 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87461d65-5012-46de-a7d9-48cd26ec3029 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:14:03.247807 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1423/2849] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:14:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:15:27.833851 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb7649-0b98-4b05-b2ae-5b2434c193d6 service placement] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:15:27.835291 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-cecb7649-0b98-4b05-b2ae-5b2434c193d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:15:27.921927 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb7649-0b98-4b05-b2ae-5b2434c193d6 service placement] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:15:27.926987 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b94f4-ab4d-408b-a5b3-8dbc4dca9541 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf9bc2fa959d4706b9f9e87509829059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:15:27.927623 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b94f4-ab4d-408b-a5b3-8dbc4dca9541 service placement] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:27.927906 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b94f4-ab4d-408b-a5b3-8dbc4dca9541 service placement] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:27.928189 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b94f4-ab4d-408b-a5b3-8dbc4dca9541 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:27.929966 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-107b94f4-ab4d-408b-a5b3-8dbc4dca9541 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:15:27.941180 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b94f4-ab4d-408b-a5b3-8dbc4dca9541 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:15:27.942779 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b94f4-ab4d-408b-a5b3-8dbc4dca9541 service placement] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:15:27.952411 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1427/2850] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:15:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:15:36.513850 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87461d65-5012-46de-a7d9-48cd26ec3029 service nova] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:15:36.515818 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-87461d65-5012-46de-a7d9-48cd26ec3029 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:15:36.613229 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87461d65-5012-46de-a7d9-48cd26ec3029 service nova] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:15:36.619391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09538a1c-4198-4a42-a215-0ca0c686a636 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:15:36.629699 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09538a1c-4198-4a42-a215-0ca0c686a636 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:36.630049 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09538a1c-4198-4a42-a215-0ca0c686a636 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:36.630317 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09538a1c-4198-4a42-a215-0ca0c686a636 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:36.639111 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-09538a1c-4198-4a42-a215-0ca0c686a636 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:15:36.653486 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09538a1c-4198-4a42-a215-0ca0c686a636 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:15:36.655483 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09538a1c-4198-4a42-a215-0ca0c686a636 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:15:36.666866 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1424/2851] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:15:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 23:15:38.963581 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b94f4-ab4d-408b-a5b3-8dbc4dca9541 service placement] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:15:38.976839 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b94f4-ab4d-408b-a5b3-8dbc4dca9541 service placement] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:15:38.982856 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457cac4-30e2-42c4-b18a-c7eedd594b10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:15:38.983648 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457cac4-30e2-42c4-b18a-c7eedd594b10 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:38.983960 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457cac4-30e2-42c4-b18a-c7eedd594b10 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:38.984265 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457cac4-30e2-42c4-b18a-c7eedd594b10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:38.986602 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a457cac4-30e2-42c4-b18a-c7eedd594b10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:15:39.004076 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457cac4-30e2-42c4-b18a-c7eedd594b10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:15:39.004840 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457cac4-30e2-42c4-b18a-c7eedd594b10 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:15:39.015874 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1428/2852] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:15:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:15:46.618882 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09538a1c-4198-4a42-a215-0ca0c686a636 service nova] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:15:46.620436 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-09538a1c-4198-4a42-a215-0ca0c686a636 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:15:46.713217 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09538a1c-4198-4a42-a215-0ca0c686a636 service nova] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:15:46.717890 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c6474-5c6b-44b6-bb00-c45839539812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:15:46.718788 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c6474-5c6b-44b6-bb00-c45839539812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:46.719065 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c6474-5c6b-44b6-bb00-c45839539812 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:46.719395 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c6474-5c6b-44b6-bb00-c45839539812 admin admin] PATH_INFO: `/v3/users/4e2656d339a74bd882c2226251ac3d36` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:46.720570 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c6474-5c6b-44b6-bb00-c45839539812 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e2656d339a74bd882c2226251ac3d36)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:15:46.723146 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c6474-5c6b-44b6-bb00-c45839539812 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:15:46.765567 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d29c6474-5c6b-44b6-bb00-c45839539812 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e2656d339a74bd882c2226251ac3d36'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:15:46.772662 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d29c6474-5c6b-44b6-bb00-c45839539812 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e2656d339a74bd882c2226251ac3d36'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:15:46.773322 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d29c6474-5c6b-44b6-bb00-c45839539812 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e2656d339a74bd882c2226251ac3d36'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:15:46.780392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d29c6474-5c6b-44b6-bb00-c45839539812 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e2656d339a74bd882c2226251ac3d36'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:15:46.781115 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d29c6474-5c6b-44b6-bb00-c45839539812 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e2656d339a74bd882c2226251ac3d36'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:15:46.797921 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1425/2853] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:15:46 2022] DELETE /identity/v3/users/4e2656d339a74bd882c2226251ac3d36 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:15:46.966205 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457cac4-30e2-42c4-b18a-c7eedd594b10 service nova] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:15:46.977053 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457cac4-30e2-42c4-b18a-c7eedd594b10 service nova] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:15:47.031953 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c61944-9065-49ee-840b-1569be0bfacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:15:47.032797 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c61944-9065-49ee-840b-1569be0bfacf admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:47.033073 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c61944-9065-49ee-840b-1569be0bfacf admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:47.033342 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c61944-9065-49ee-840b-1569be0bfacf admin admin] PATH_INFO: `/v3/projects/130ce8965a764f6d8910c2bba4561534` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:47.034568 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c61944-9065-49ee-840b-1569be0bfacf admin admin] RBAC: Authorizing `identity:delete_project(project_id=130ce8965a764f6d8910c2bba4561534)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:15:47.036450 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c61944-9065-49ee-840b-1569be0bfacf admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:15:47.083543 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d9c61944-9065-49ee-840b-1569be0bfacf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '130ce8965a764f6d8910c2bba4561534', 'request_id': 'req-d9c61944-9065-49ee-840b-1569be0bfacf'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:15:47.088738 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d9c61944-9065-49ee-840b-1569be0bfacf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '130ce8965a764f6d8910c2bba4561534', 'request_id': 'req-d9c61944-9065-49ee-840b-1569be0bfacf'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:15:47.094709 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d9c61944-9065-49ee-840b-1569be0bfacf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '130ce8965a764f6d8910c2bba4561534', 'request_id': 'req-d9c61944-9065-49ee-840b-1569be0bfacf'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:15:47.106297 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d9c61944-9065-49ee-840b-1569be0bfacf admin admin] The token cache is being invalidate because project 130ce8965a764f6d8910c2bba4561534 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:15:47.106581 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d9c61944-9065-49ee-840b-1569be0bfacf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:15:47.107195 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d9c61944-9065-49ee-840b-1569be0bfacf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:15:47.108738 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1429/2854] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:15:46 2022] DELETE /identity/v3/projects/130ce8965a764f6d8910c2bba4561534 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:15:47.112823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c6474-5c6b-44b6-bb00-c45839539812 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:15:47.114249 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d29c6474-5c6b-44b6-bb00-c45839539812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:15:47.127134 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c6474-5c6b-44b6-bb00-c45839539812 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:15:47.181081 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79ddb0-aebf-437c-b1fe-11c79ed69cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:15:47.181859 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79ddb0-aebf-437c-b1fe-11c79ed69cc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:47.182153 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79ddb0-aebf-437c-b1fe-11c79ed69cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:47.182424 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79ddb0-aebf-437c-b1fe-11c79ed69cc7 admin admin] PATH_INFO: `/v3/users/360edf7a5cd4488fac988060b7076405` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:47.183573 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb79ddb0-aebf-437c-b1fe-11c79ed69cc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=360edf7a5cd4488fac988060b7076405)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:15:47.185471 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb79ddb0-aebf-437c-b1fe-11c79ed69cc7 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:15:47.229382 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eb79ddb0-aebf-437c-b1fe-11c79ed69cc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '360edf7a5cd4488fac988060b7076405'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:15:47.236051 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eb79ddb0-aebf-437c-b1fe-11c79ed69cc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '360edf7a5cd4488fac988060b7076405'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:15:47.236700 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eb79ddb0-aebf-437c-b1fe-11c79ed69cc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '360edf7a5cd4488fac988060b7076405'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:15:47.243771 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eb79ddb0-aebf-437c-b1fe-11c79ed69cc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '360edf7a5cd4488fac988060b7076405'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:15:47.244388 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eb79ddb0-aebf-437c-b1fe-11c79ed69cc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '360edf7a5cd4488fac988060b7076405'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:15:47.260727 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1426/2855] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:15:47 2022] DELETE /identity/v3/users/360edf7a5cd4488fac988060b7076405 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:15:47.430940 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c61944-9065-49ee-840b-1569be0bfacf admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:15:47.432576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d9c61944-9065-49ee-840b-1569be0bfacf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:15:47.448326 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c61944-9065-49ee-840b-1569be0bfacf admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:15:47.502307 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c28f0-29c3-4188-9fb6-d1893d25476d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:15:47.503169 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c28f0-29c3-4188-9fb6-d1893d25476d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:47.503442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c28f0-29c3-4188-9fb6-d1893d25476d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:47.503708 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c28f0-29c3-4188-9fb6-d1893d25476d admin admin] PATH_INFO: `/v3/projects/96cc6132d1dd4be8aba03b926a5afef7` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:47.504864 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c28f0-29c3-4188-9fb6-d1893d25476d admin admin] RBAC: Authorizing `identity:delete_project(project_id=96cc6132d1dd4be8aba03b926a5afef7)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:15:47.506809 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c28f0-29c3-4188-9fb6-d1893d25476d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:15:47.550649 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c42c28f0-29c3-4188-9fb6-d1893d25476d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96cc6132d1dd4be8aba03b926a5afef7', 'request_id': 'req-c42c28f0-29c3-4188-9fb6-d1893d25476d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:15:47.555735 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c42c28f0-29c3-4188-9fb6-d1893d25476d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96cc6132d1dd4be8aba03b926a5afef7', 'request_id': 'req-c42c28f0-29c3-4188-9fb6-d1893d25476d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:15:47.561572 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c42c28f0-29c3-4188-9fb6-d1893d25476d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96cc6132d1dd4be8aba03b926a5afef7', 'request_id': 'req-c42c28f0-29c3-4188-9fb6-d1893d25476d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:15:47.574041 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c42c28f0-29c3-4188-9fb6-d1893d25476d admin admin] The token cache is being invalidate because project 96cc6132d1dd4be8aba03b926a5afef7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:15:47.574357 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c42c28f0-29c3-4188-9fb6-d1893d25476d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:15:47.574969 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c42c28f0-29c3-4188-9fb6-d1893d25476d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:15:47.576472 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1430/2856] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:15:47 2022] DELETE /identity/v3/projects/96cc6132d1dd4be8aba03b926a5afef7 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:15:47.582075 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dac154-c5f7-4651-bd9b-79f0020d10fd None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:47.582392 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dac154-c5f7-4651-bd9b-79f0020d10fd None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:47.582668 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dac154-c5f7-4651-bd9b-79f0020d10fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:47.636845 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-c4dac154-c5f7-4651-bd9b-79f0020d10fd None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:15:47.638091 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c4dac154-c5f7-4651-bd9b-79f0020d10fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:15:47.696913 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1427/2857] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:15:47.706300 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3be762-c9fb-4da1-9072-4841b0ffcf66 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:47.706622 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3be762-c9fb-4da1-9072-4841b0ffcf66 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:47.706899 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3be762-c9fb-4da1-9072-4841b0ffcf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:47.726926 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-6d3be762-c9fb-4da1-9072-4841b0ffcf66 None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:15:47.728207 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6d3be762-c9fb-4da1-9072-4841b0ffcf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:15:47.737564 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1431/2858] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:15:47.741720 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dac154-c5f7-4651-bd9b-79f0020d10fd None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:15:47.751178 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dac154-c5f7-4651-bd9b-79f0020d10fd None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:15:47.756245 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e57193-fc82-450d-a887-2b3416306811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:15:47.756895 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e57193-fc82-450d-a887-2b3416306811 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:47.757178 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e57193-fc82-450d-a887-2b3416306811 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:47.757454 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e57193-fc82-450d-a887-2b3416306811 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:47.758203 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e57193-fc82-450d-a887-2b3416306811 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 23:15:47.758510 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e57193-fc82-450d-a887-2b3416306811 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:15:47.760098 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e57193-fc82-450d-a887-2b3416306811 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:15:47.772179 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1428/2859] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 9 23:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 23:15:47.776482 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3be762-c9fb-4da1-9072-4841b0ffcf66 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:15:47.786521 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3be762-c9fb-4da1-9072-4841b0ffcf66 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:15:47.794835 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887e6e7-7057-4ed1-a7d9-8e2da19207d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:15:47.795562 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887e6e7-7057-4ed1-a7d9-8e2da19207d2 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:47.795838 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887e6e7-7057-4ed1-a7d9-8e2da19207d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:47.796110 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887e6e7-7057-4ed1-a7d9-8e2da19207d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:47.797193 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887e6e7-7057-4ed1-a7d9-8e2da19207d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:15:47.800413 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887e6e7-7057-4ed1-a7d9-8e2da19207d2 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:15:47.824898 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1432/2860] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:15:47 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:15:47.829155 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e57193-fc82-450d-a887-2b3416306811 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:15:47.838900 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e57193-fc82-450d-a887-2b3416306811 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:15:47.893481 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692bb67-50bb-4c59-9071-5aa67de1e723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:15:47.894159 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692bb67-50bb-4c59-9071-5aa67de1e723 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:47.894441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692bb67-50bb-4c59-9071-5aa67de1e723 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:47.894715 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692bb67-50bb-4c59-9071-5aa67de1e723 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:47.895386 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692bb67-50bb-4c59-9071-5aa67de1e723 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:15:47.897293 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692bb67-50bb-4c59-9071-5aa67de1e723 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:15:47.916383 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1429/2861] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:15:47 2022] POST /identity/v3/users => generated 290 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:15:47.920442 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887e6e7-7057-4ed1-a7d9-8e2da19207d2 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:15:47.931255 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887e6e7-7057-4ed1-a7d9-8e2da19207d2 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:15:47.935887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1f4d2-81c8-4ad9-a6e3-7dff8643ea47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:15:47.936487 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1f4d2-81c8-4ad9-a6e3-7dff8643ea47 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:47.936785 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1f4d2-81c8-4ad9-a6e3-7dff8643ea47 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:47.937062 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1f4d2-81c8-4ad9-a6e3-7dff8643ea47 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:47.937778 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b1f4d2-81c8-4ad9-a6e3-7dff8643ea47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:15:47.939403 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b1f4d2-81c8-4ad9-a6e3-7dff8643ea47 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:15:47.949790 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1433/2862] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:15:47.953613 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692bb67-50bb-4c59-9071-5aa67de1e723 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:15:47.963946 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692bb67-50bb-4c59-9071-5aa67de1e723 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:15:47.968579 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720993a-0698-422f-aec4-59d1c73ebf24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:15:47.969226 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720993a-0698-422f-aec4-59d1c73ebf24 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:47.969506 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720993a-0698-422f-aec4-59d1c73ebf24 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:47.969778 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720993a-0698-422f-aec4-59d1c73ebf24 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:47.970535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720993a-0698-422f-aec4-59d1c73ebf24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:15:47.972746 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720993a-0698-422f-aec4-59d1c73ebf24 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:15:47.982623 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1430/2863] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:15:47.986563 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1f4d2-81c8-4ad9-a6e3-7dff8643ea47 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:15:47.996283 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1f4d2-81c8-4ad9-a6e3-7dff8643ea47 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:15:48.001144 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5513f-ce0a-410c-81b5-725d29bc3a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:15:48.001996 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5513f-ce0a-410c-81b5-725d29bc3a55 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:48.002309 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5513f-ce0a-410c-81b5-725d29bc3a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:48.002626 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5513f-ce0a-410c-81b5-725d29bc3a55 admin admin] PATH_INFO: `/v3/projects/d557435c4af9467999e66a94c6929e11/users/069d3a8dd790459db3bcfea55f8e19f4/roles/3631750e6bf34730aa3557cd90aa6108` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:48.016366 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5513f-ce0a-410c-81b5-725d29bc3a55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d557435c4af9467999e66a94c6929e11, user_id=069d3a8dd790459db3bcfea55f8e19f4, role_id=3631750e6bf34730aa3557cd90aa6108)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:15:48.019581 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5513f-ce0a-410c-81b5-725d29bc3a55 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:15:48.031846 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1434/2864] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:15:47 2022] PUT /identity/v3/projects/d557435c4af9467999e66a94c6929e11/users/069d3a8dd790459db3bcfea55f8e19f4/roles/3631750e6bf34730aa3557cd90aa6108 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:15:48.037454 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20de90-6621-4e9b-9fba-d9a745ac61ab None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:48.037769 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20de90-6621-4e9b-9fba-d9a745ac61ab None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:48.038047 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20de90-6621-4e9b-9fba-d9a745ac61ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:48.055239 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-7c20de90-6621-4e9b-9fba-d9a745ac61ab None None] MFA Rules not processed for user `873ba60d483a41a4b36c04c1aa8b34ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:15:48.056433 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7c20de90-6621-4e9b-9fba-d9a745ac61ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:15:48.112555 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1431/2865] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:15:48.122534 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5513f-ce0a-410c-81b5-725d29bc3a55 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:15:48.124237 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-45d5513f-ce0a-410c-81b5-725d29bc3a55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:15:48.199471 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5513f-ce0a-410c-81b5-725d29bc3a55 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:15:48.203051 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533ac01-8a65-4751-862a-68a170f27e85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:15:48.203693 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533ac01-8a65-4751-862a-68a170f27e85 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:48.203972 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533ac01-8a65-4751-862a-68a170f27e85 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:48.204246 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533ac01-8a65-4751-862a-68a170f27e85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:48.213707 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7533ac01-8a65-4751-862a-68a170f27e85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:15:48.215179 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7533ac01-8a65-4751-862a-68a170f27e85 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:15:48.225062 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1435/2866] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:15:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:15:52.152079 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99957fe-e524-46c1-b1ba-56a3d3af1639 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:52.152600 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99957fe-e524-46c1-b1ba-56a3d3af1639 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:52.152600 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99957fe-e524-46c1-b1ba-56a3d3af1639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:52.180522 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d99957fe-e524-46c1-b1ba-56a3d3af1639 None None] MFA Rules not processed for user `069d3a8dd790459db3bcfea55f8e19f4`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:15:52.181823 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d99957fe-e524-46c1-b1ba-56a3d3af1639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:15:52.257654 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1432/2867] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:15:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:15:52.258141 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533ac01-8a65-4751-862a-68a170f27e85 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:15:52.268654 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533ac01-8a65-4751-862a-68a170f27e85 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:15:52.278041 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ebb5a-dd3e-45d8-a7e8-793a16906eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:15:52.278751 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ebb5a-dd3e-45d8-a7e8-793a16906eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:52.279032 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ebb5a-dd3e-45d8-a7e8-793a16906eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:52.279312 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ebb5a-dd3e-45d8-a7e8-793a16906eb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:52.280004 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ebb5a-dd3e-45d8-a7e8-793a16906eb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:15:52.281894 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ebb5a-dd3e-45d8-a7e8-793a16906eb8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:15:52.312375 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1436/2868] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 9 23:15:52 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 23:15:52.317472 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99957fe-e524-46c1-b1ba-56a3d3af1639 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:15:52.328854 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99957fe-e524-46c1-b1ba-56a3d3af1639 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:15:52.391923 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767939da-c4f5-4a13-aa6a-70fbd1d11daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:15:52.392635 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767939da-c4f5-4a13-aa6a-70fbd1d11daa admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:52.392916 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767939da-c4f5-4a13-aa6a-70fbd1d11daa admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:52.393236 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767939da-c4f5-4a13-aa6a-70fbd1d11daa admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:52.393928 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767939da-c4f5-4a13-aa6a-70fbd1d11daa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:15:52.395935 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767939da-c4f5-4a13-aa6a-70fbd1d11daa admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:15:52.417686 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1433/2869] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 9 23:15:52 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 23:15:52.421666 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ebb5a-dd3e-45d8-a7e8-793a16906eb8 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:15:52.432699 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ebb5a-dd3e-45d8-a7e8-793a16906eb8 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:15:52.437576 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ac480-3552-4120-91b7-ceddda726347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:15:52.438288 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ac480-3552-4120-91b7-ceddda726347 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:52.438573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ac480-3552-4120-91b7-ceddda726347 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:52.438853 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ac480-3552-4120-91b7-ceddda726347 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:52.439574 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ac480-3552-4120-91b7-ceddda726347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:15:52.441496 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ac480-3552-4120-91b7-ceddda726347 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:15:52.452109 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1437/2870] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:15:52.456788 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767939da-c4f5-4a13-aa6a-70fbd1d11daa admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:15:52.467339 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767939da-c4f5-4a13-aa6a-70fbd1d11daa admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:15:52.472507 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc2a6c-c75b-451c-9d4a-47bc2908d549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:15:52.473394 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc2a6c-c75b-451c-9d4a-47bc2908d549 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:52.473677 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc2a6c-c75b-451c-9d4a-47bc2908d549 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:52.473953 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc2a6c-c75b-451c-9d4a-47bc2908d549 admin admin] PATH_INFO: `/v3/projects/7984ca6235ee49c6bc64239f528fb36e/users/90cc580a6c084c28bc8538f9d889139a/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:52.489966 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc2a6c-c75b-451c-9d4a-47bc2908d549 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7984ca6235ee49c6bc64239f528fb36e, user_id=90cc580a6c084c28bc8538f9d889139a, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:15:52.492693 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc2a6c-c75b-451c-9d4a-47bc2908d549 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:15:52.510028 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1434/2871] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:15:52 2022] PUT /identity/v3/projects/7984ca6235ee49c6bc64239f528fb36e/users/90cc580a6c084c28bc8538f9d889139a/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 23:15:52.511070 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ac480-3552-4120-91b7-ceddda726347 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:15:52.522182 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ac480-3552-4120-91b7-ceddda726347 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:15:52.582990 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eaa708-47db-4d8f-bf36-285e49484645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:15:52.583579 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eaa708-47db-4d8f-bf36-285e49484645 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:52.583859 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eaa708-47db-4d8f-bf36-285e49484645 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:52.584133 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eaa708-47db-4d8f-bf36-285e49484645 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:52.584851 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eaa708-47db-4d8f-bf36-285e49484645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:15:52.586519 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eaa708-47db-4d8f-bf36-285e49484645 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:15:52.597086 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1438/2872] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 9 23:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 23:15:52.601226 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc2a6c-c75b-451c-9d4a-47bc2908d549 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:15:52.612602 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc2a6c-c75b-451c-9d4a-47bc2908d549 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:15:52.617449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11adb01-6175-4923-8dc1-e0993dafdd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:15:52.618376 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11adb01-6175-4923-8dc1-e0993dafdd2d admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:52.618660 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11adb01-6175-4923-8dc1-e0993dafdd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:52.618937 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11adb01-6175-4923-8dc1-e0993dafdd2d admin admin] PATH_INFO: `/v3/projects/7984ca6235ee49c6bc64239f528fb36e/users/90cc580a6c084c28bc8538f9d889139a/roles/bda550492f4f4c5c83f52fa89fc18a7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:52.620917 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11adb01-6175-4923-8dc1-e0993dafdd2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7984ca6235ee49c6bc64239f528fb36e, user_id=90cc580a6c084c28bc8538f9d889139a, role_id=bda550492f4f4c5c83f52fa89fc18a7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:15:52.623795 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11adb01-6175-4923-8dc1-e0993dafdd2d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:15:52.637037 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1435/2873] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 9 23:15:52 2022] PUT /identity/v3/projects/7984ca6235ee49c6bc64239f528fb36e/users/90cc580a6c084c28bc8538f9d889139a/roles/bda550492f4f4c5c83f52fa89fc18a7d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:15:56.853739 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a75d4-39a7-41b1-a093-ee51dc784b9d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:56.854616 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a75d4-39a7-41b1-a093-ee51dc784b9d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:56.854616 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a75d4-39a7-41b1-a093-ee51dc784b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:56.881708 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-926a75d4-39a7-41b1-a093-ee51dc784b9d None None] MFA Rules not processed for user `90cc580a6c084c28bc8538f9d889139a`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:15:56.883607 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-926a75d4-39a7-41b1-a093-ee51dc784b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:15:56.952872 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1439/2874] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:15:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:15:56.966235 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11adb01-6175-4923-8dc1-e0993dafdd2d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:15:56.968040 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b11adb01-6175-4923-8dc1-e0993dafdd2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:15:57.041445 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11adb01-6175-4923-8dc1-e0993dafdd2d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:15:57.046966 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d5198-0aa1-42c5-9520-7dd958c92fe5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:15:57.047610 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d5198-0aa1-42c5-9520-7dd958c92fe5 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:15:57.047891 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d5198-0aa1-42c5-9520-7dd958c92fe5 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:15:57.048165 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d5198-0aa1-42c5-9520-7dd958c92fe5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:15:57.102928 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d5198-0aa1-42c5-9520-7dd958c92fe5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:15:57.104722 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d5198-0aa1-42c5-9520-7dd958c92fe5 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:15:57.113898 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1436/2875] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:15:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:16:14.422637 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2690f1-5614-4992-95ac-1ce22ba216cd None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:16:14.423140 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2690f1-5614-4992-95ac-1ce22ba216cd None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:16:14.423253 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2690f1-5614-4992-95ac-1ce22ba216cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:16:14.441331 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-0b2690f1-5614-4992-95ac-1ce22ba216cd None None] MFA Rules not processed for user `069d3a8dd790459db3bcfea55f8e19f4`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:16:14.442561 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0b2690f1-5614-4992-95ac-1ce22ba216cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:16:14.449641 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1440/2876] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:16:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:16:14.751437 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d5198-0aa1-42c5-9520-7dd958c92fe5 service nova] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:16:14.803872 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d5198-0aa1-42c5-9520-7dd958c92fe5 service nova] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:16:14.807484 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c4c81-bd96-42f3-bbc1-93885d915a6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:16:14.808109 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c4c81-bd96-42f3-bbc1-93885d915a6f service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:16:14.808385 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c4c81-bd96-42f3-bbc1-93885d915a6f service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:16:14.808652 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c4c81-bd96-42f3-bbc1-93885d915a6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:16:14.810449 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c56c4c81-bd96-42f3-bbc1-93885d915a6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:16:14.819614 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c4c81-bd96-42f3-bbc1-93885d915a6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:16:14.821020 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c4c81-bd96-42f3-bbc1-93885d915a6f service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:16:14.828701 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1437/2877] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:16:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:16:31.036175 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2690f1-5614-4992-95ac-1ce22ba216cd None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:16:31.046925 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2690f1-5614-4992-95ac-1ce22ba216cd None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:16:31.050526 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac1486-2a6f-43d5-8194-60966c5b0174 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:16:31.051160 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac1486-2a6f-43d5-8194-60966c5b0174 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:16:31.051438 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac1486-2a6f-43d5-8194-60966c5b0174 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:16:31.051705 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac1486-2a6f-43d5-8194-60966c5b0174 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:16:31.060760 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac1486-2a6f-43d5-8194-60966c5b0174 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:16:31.062272 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac1486-2a6f-43d5-8194-60966c5b0174 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:16:31.072796 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1441/2878] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:16:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:18:29.007463 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c4c81-bd96-42f3-bbc1-93885d915a6f service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:18:29.008957 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c56c4c81-bd96-42f3-bbc1-93885d915a6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:18:29.019122 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c4c81-bd96-42f3-bbc1-93885d915a6f service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:18:29.022731 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ab7b8-db2a-4f1c-8a4d-3e454f2074fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:18:29.023363 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ab7b8-db2a-4f1c-8a4d-3e454f2074fc service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:18:29.023642 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ab7b8-db2a-4f1c-8a4d-3e454f2074fc service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:18:29.023914 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ab7b8-db2a-4f1c-8a4d-3e454f2074fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:18:29.025634 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-927ab7b8-db2a-4f1c-8a4d-3e454f2074fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:18:29.038846 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ab7b8-db2a-4f1c-8a4d-3e454f2074fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:18:29.040252 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ab7b8-db2a-4f1c-8a4d-3e454f2074fc service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:18:29.050932 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1438/2879] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:18:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 23:18:30.224344 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac1486-2a6f-43d5-8194-60966c5b0174 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:18:30.233630 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac1486-2a6f-43d5-8194-60966c5b0174 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:18:30.237203 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f27e6d-6177-418a-bfaa-0567995d0c1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:18:30.238095 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f27e6d-6177-418a-bfaa-0567995d0c1f service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:18:30.238408 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f27e6d-6177-418a-bfaa-0567995d0c1f service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:18:30.238684 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f27e6d-6177-418a-bfaa-0567995d0c1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:18:30.241067 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e4f27e6d-6177-418a-bfaa-0567995d0c1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:18:30.254055 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f27e6d-6177-418a-bfaa-0567995d0c1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:18:30.255493 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f27e6d-6177-418a-bfaa-0567995d0c1f service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:18:30.266201 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1442/2880] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:18:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:18:32.705985 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd088e9-ae21-4ef1-8d70-68e4cf3acac3 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:18:32.706505 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd088e9-ae21-4ef1-8d70-68e4cf3acac3 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:18:32.706604 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd088e9-ae21-4ef1-8d70-68e4cf3acac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:18:32.731061 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-dfd088e9-ae21-4ef1-8d70-68e4cf3acac3 None None] MFA Rules not processed for user `069d3a8dd790459db3bcfea55f8e19f4`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:18:32.732334 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-dfd088e9-ae21-4ef1-8d70-68e4cf3acac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:18:32.740332 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1439/2881] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:18:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:18:39.431785 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f27e6d-6177-418a-bfaa-0567995d0c1f service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:18:39.440851 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f27e6d-6177-418a-bfaa-0567995d0c1f service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:18:39.444581 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3053f2-e40b-4a13-bb9f-3ea73b6ec58a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:18:39.445226 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3053f2-e40b-4a13-bb9f-3ea73b6ec58a service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:18:39.445504 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3053f2-e40b-4a13-bb9f-3ea73b6ec58a service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:18:39.445786 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3053f2-e40b-4a13-bb9f-3ea73b6ec58a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:18:39.447588 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ee3053f2-e40b-4a13-bb9f-3ea73b6ec58a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:18:39.457887 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3053f2-e40b-4a13-bb9f-3ea73b6ec58a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:18:39.459961 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3053f2-e40b-4a13-bb9f-3ea73b6ec58a service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:18:39.469033 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1443/2882] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:18:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:20:30.279664 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14404fd3-ebdb-4696-8dc6-578084f7b313 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:20:30.280208 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14404fd3-ebdb-4696-8dc6-578084f7b313 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:20:30.280208 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14404fd3-ebdb-4696-8dc6-578084f7b313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:20:30.297730 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-14404fd3-ebdb-4696-8dc6-578084f7b313 None None] MFA Rules not processed for user `069d3a8dd790459db3bcfea55f8e19f4`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 23:20:30.298931 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-14404fd3-ebdb-4696-8dc6-578084f7b313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:20:30.306292 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1440/2883] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 9 23:20:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 23:20:30.626097 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3053f2-e40b-4a13-bb9f-3ea73b6ec58a service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:20:30.635396 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3053f2-e40b-4a13-bb9f-3ea73b6ec58a service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:20:30.639059 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2b28d-d196-430d-bd2b-d2343db55b36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:20:30.639690 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2b28d-d196-430d-bd2b-d2343db55b36 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:20:30.639969 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2b28d-d196-430d-bd2b-d2343db55b36 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:20:30.640243 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2b28d-d196-430d-bd2b-d2343db55b36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:20:30.641973 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-86b2b28d-d196-430d-bd2b-d2343db55b36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:20:30.651469 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b2b28d-d196-430d-bd2b-d2343db55b36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:20:30.652886 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b2b28d-d196-430d-bd2b-d2343db55b36 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:20:30.660731 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1444/2884] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:20:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:20:31.797121 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14404fd3-ebdb-4696-8dc6-578084f7b313 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:20:31.798535 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-14404fd3-ebdb-4696-8dc6-578084f7b313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:20:31.866551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14404fd3-ebdb-4696-8dc6-578084f7b313 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:20:31.871428 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420cc217-242e-4582-9815-e3ed95f335bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf9bc2fa959d4706b9f9e87509829059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:20:31.872055 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420cc217-242e-4582-9815-e3ed95f335bc service placement] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:20:31.872328 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420cc217-242e-4582-9815-e3ed95f335bc service placement] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:20:31.872597 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420cc217-242e-4582-9815-e3ed95f335bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:20:31.874322 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-420cc217-242e-4582-9815-e3ed95f335bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:20:31.885092 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420cc217-242e-4582-9815-e3ed95f335bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:20:31.886597 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420cc217-242e-4582-9815-e3ed95f335bc service placement] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:20:31.897296 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1441/2885] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:20:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:20:32.115029 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2b28d-d196-430d-bd2b-d2343db55b36 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:20:32.116422 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-86b2b28d-d196-430d-bd2b-d2343db55b36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:20:32.131086 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2b28d-d196-430d-bd2b-d2343db55b36 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:20:32.136952 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2216a-9eda-4f1c-9772-b903b28c10a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf9bc2fa959d4706b9f9e87509829059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:20:32.137805 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2216a-9eda-4f1c-9772-b903b28c10a9 service placement] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:20:32.138089 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2216a-9eda-4f1c-9772-b903b28c10a9 service placement] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:20:32.138396 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2216a-9eda-4f1c-9772-b903b28c10a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:20:32.140142 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-92e2216a-9eda-4f1c-9772-b903b28c10a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:20:32.153993 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e2216a-9eda-4f1c-9772-b903b28c10a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:20:32.155601 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e2216a-9eda-4f1c-9772-b903b28c10a9 service placement] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:20:32.165942 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1445/2886] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:20:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:20:45.160776 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420cc217-242e-4582-9815-e3ed95f335bc service placement] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:20:45.174099 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420cc217-242e-4582-9815-e3ed95f335bc service placement] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:20:45.180349 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e54371-7452-4e7a-beae-f2ba4e9c9bbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:20:45.180992 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e54371-7452-4e7a-beae-f2ba4e9c9bbb service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:20:45.181270 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e54371-7452-4e7a-beae-f2ba4e9c9bbb service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:20:45.181541 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e54371-7452-4e7a-beae-f2ba4e9c9bbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:20:45.183296 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e2e54371-7452-4e7a-beae-f2ba4e9c9bbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:20:45.195642 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e54371-7452-4e7a-beae-f2ba4e9c9bbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:20:45.197216 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e54371-7452-4e7a-beae-f2ba4e9c9bbb service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:20:45.207989 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1442/2887] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:20:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:20:49.310051 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2216a-9eda-4f1c-9772-b903b28c10a9 service placement] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:20:49.322751 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2216a-9eda-4f1c-9772-b903b28c10a9 service placement] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:20:49.328257 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cbcf2-8237-43a7-879a-54a788890f73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:20:49.328893 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cbcf2-8237-43a7-879a-54a788890f73 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:20:49.329174 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cbcf2-8237-43a7-879a-54a788890f73 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:20:49.329447 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cbcf2-8237-43a7-879a-54a788890f73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:20:49.331250 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-285cbcf2-8237-43a7-879a-54a788890f73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:20:49.343266 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285cbcf2-8237-43a7-879a-54a788890f73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:20:49.344823 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285cbcf2-8237-43a7-879a-54a788890f73 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:20:49.356758 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1446/2888] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:20:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:22:17.708431 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e54371-7452-4e7a-beae-f2ba4e9c9bbb service nova] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:22:17.718724 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e54371-7452-4e7a-beae-f2ba4e9c9bbb service nova] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:22:17.722770 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65061605-17c0-4e19-a88f-f33e056c1c1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:22:17.723396 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65061605-17c0-4e19-a88f-f33e056c1c1d service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:22:17.723700 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65061605-17c0-4e19-a88f-f33e056c1c1d service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:22:17.723972 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65061605-17c0-4e19-a88f-f33e056c1c1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:22:17.732107 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65061605-17c0-4e19-a88f-f33e056c1c1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:22:17.733514 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65061605-17c0-4e19-a88f-f33e056c1c1d service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:22:17.742422 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1443/2889] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:22:17 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:22:27.561149 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cbcf2-8237-43a7-879a-54a788890f73 service nova] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:22:27.572977 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cbcf2-8237-43a7-879a-54a788890f73 service nova] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:22:27.579423 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3d1bf-da99-4b91-a8f1-763f9f05d75c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fcd7cc86134ce488c7dd75dfb174b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:22:27.580240 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3d1bf-da99-4b91-a8f1-763f9f05d75c service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:22:27.580573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3d1bf-da99-4b91-a8f1-763f9f05d75c service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:22:27.580814 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3d1bf-da99-4b91-a8f1-763f9f05d75c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:22:27.582619 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b6f3d1bf-da99-4b91-a8f1-763f9f05d75c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:22:27.595573 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f3d1bf-da99-4b91-a8f1-763f9f05d75c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:22:27.597168 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f3d1bf-da99-4b91-a8f1-763f9f05d75c service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:22:27.608726 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1447/2890] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:22:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:22:29.897904 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65061605-17c0-4e19-a88f-f33e056c1c1d service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:22:29.906562 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65061605-17c0-4e19-a88f-f33e056c1c1d service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:22:29.910099 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01546a5-1862-41f9-b20f-3afaebc48d87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96fd3f6dd9e74ba7a4ad159e43e8ae96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:22:29.910755 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01546a5-1862-41f9-b20f-3afaebc48d87 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:22:29.911033 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01546a5-1862-41f9-b20f-3afaebc48d87 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:22:29.911330 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01546a5-1862-41f9-b20f-3afaebc48d87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:22:29.969370 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01546a5-1862-41f9-b20f-3afaebc48d87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:22:29.970832 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01546a5-1862-41f9-b20f-3afaebc48d87 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:22:29.980480 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1444/2891] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:22:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 23:22:36.304708 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3d1bf-da99-4b91-a8f1-763f9f05d75c service nova] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:22:36.315944 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3d1bf-da99-4b91-a8f1-763f9f05d75c service nova] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:22:36.321639 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c2280-24b2-4a5c-8c13-6003f7c28260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:22:36.322608 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c2280-24b2-4a5c-8c13-6003f7c28260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:22:36.322889 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c2280-24b2-4a5c-8c13-6003f7c28260 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:22:36.323053 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c2280-24b2-4a5c-8c13-6003f7c28260 admin admin] PATH_INFO: `/v3/users/069d3a8dd790459db3bcfea55f8e19f4` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:22:36.324226 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c2280-24b2-4a5c-8c13-6003f7c28260 admin admin] RBAC: Authorizing `identity:delete_user(user_id=069d3a8dd790459db3bcfea55f8e19f4)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:22:36.326156 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c2280-24b2-4a5c-8c13-6003f7c28260 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:22:36.374036 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-1a0c2280-24b2-4a5c-8c13-6003f7c28260 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '069d3a8dd790459db3bcfea55f8e19f4'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:22:36.380871 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-1a0c2280-24b2-4a5c-8c13-6003f7c28260 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '069d3a8dd790459db3bcfea55f8e19f4'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:22:36.388169 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-1a0c2280-24b2-4a5c-8c13-6003f7c28260 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '069d3a8dd790459db3bcfea55f8e19f4'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:22:36.388784 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-1a0c2280-24b2-4a5c-8c13-6003f7c28260 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '069d3a8dd790459db3bcfea55f8e19f4'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:22:36.400863 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-1a0c2280-24b2-4a5c-8c13-6003f7c28260 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '069d3a8dd790459db3bcfea55f8e19f4'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:22:36.406842 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1448/2892] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:22:36 2022] DELETE /identity/v3/users/069d3a8dd790459db3bcfea55f8e19f4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:22:36.581409 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01546a5-1862-41f9-b20f-3afaebc48d87 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:22:36.592418 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01546a5-1862-41f9-b20f-3afaebc48d87 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:22:36.648157 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e77d17-2482-42a8-b474-4d4322742846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:22:36.649007 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e77d17-2482-42a8-b474-4d4322742846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:22:36.649283 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e77d17-2482-42a8-b474-4d4322742846 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:22:36.649554 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e77d17-2482-42a8-b474-4d4322742846 admin admin] PATH_INFO: `/v3/projects/d557435c4af9467999e66a94c6929e11` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:22:36.650754 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e77d17-2482-42a8-b474-4d4322742846 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d557435c4af9467999e66a94c6929e11)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:22:36.652649 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e77d17-2482-42a8-b474-4d4322742846 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:22:36.697856 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-48e77d17-2482-42a8-b474-4d4322742846 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd557435c4af9467999e66a94c6929e11', 'request_id': 'req-48e77d17-2482-42a8-b474-4d4322742846'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:22:36.704165 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-48e77d17-2482-42a8-b474-4d4322742846 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd557435c4af9467999e66a94c6929e11', 'request_id': 'req-48e77d17-2482-42a8-b474-4d4322742846'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:22:36.709441 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-48e77d17-2482-42a8-b474-4d4322742846 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd557435c4af9467999e66a94c6929e11', 'request_id': 'req-48e77d17-2482-42a8-b474-4d4322742846'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:22:36.721461 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-48e77d17-2482-42a8-b474-4d4322742846 admin admin] The token cache is being invalidate because project d557435c4af9467999e66a94c6929e11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:22:36.721749 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-48e77d17-2482-42a8-b474-4d4322742846 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:22:36.722391 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-48e77d17-2482-42a8-b474-4d4322742846 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:22:36.723906 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1445/2893] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:22:36 2022] DELETE /identity/v3/projects/d557435c4af9467999e66a94c6929e11 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:22:36.728619 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c2280-24b2-4a5c-8c13-6003f7c28260 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:22:36.730018 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1a0c2280-24b2-4a5c-8c13-6003f7c28260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:22:36.743433 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c2280-24b2-4a5c-8c13-6003f7c28260 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:22:36.796784 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb1191-1fd0-4ff4-be96-63da8bd09fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:22:36.797644 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb1191-1fd0-4ff4-be96-63da8bd09fcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:22:36.797923 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb1191-1fd0-4ff4-be96-63da8bd09fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:22:36.798233 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb1191-1fd0-4ff4-be96-63da8bd09fcc admin admin] PATH_INFO: `/v3/users/90cc580a6c084c28bc8538f9d889139a` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:22:36.799396 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fb1191-1fd0-4ff4-be96-63da8bd09fcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=90cc580a6c084c28bc8538f9d889139a)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:22:36.801290 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fb1191-1fd0-4ff4-be96-63da8bd09fcc admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:22:36.844320 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-54fb1191-1fd0-4ff4-be96-63da8bd09fcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90cc580a6c084c28bc8538f9d889139a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:22:36.851463 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-54fb1191-1fd0-4ff4-be96-63da8bd09fcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90cc580a6c084c28bc8538f9d889139a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:22:36.858564 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-54fb1191-1fd0-4ff4-be96-63da8bd09fcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90cc580a6c084c28bc8538f9d889139a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:22:36.859168 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-54fb1191-1fd0-4ff4-be96-63da8bd09fcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90cc580a6c084c28bc8538f9d889139a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:22:36.870841 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-54fb1191-1fd0-4ff4-be96-63da8bd09fcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90cc580a6c084c28bc8538f9d889139a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:22:36.876417 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1449/2894] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 9 23:22:36 2022] DELETE /identity/v3/users/90cc580a6c084c28bc8538f9d889139a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:22:37.291779 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e77d17-2482-42a8-b474-4d4322742846 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:22:37.293378 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-48e77d17-2482-42a8-b474-4d4322742846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:22:37.310795 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e77d17-2482-42a8-b474-4d4322742846 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:22:37.366142 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de916a5-ef65-489d-86e6-b638abb7f841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '873ba60d483a41a4b36c04c1aa8b34ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4202d89a17374bc0a3d992386d254d4c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:22:37.366997 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de916a5-ef65-489d-86e6-b638abb7f841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:22:37.367278 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de916a5-ef65-489d-86e6-b638abb7f841 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:22:37.367551 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de916a5-ef65-489d-86e6-b638abb7f841 admin admin] PATH_INFO: `/v3/projects/7984ca6235ee49c6bc64239f528fb36e` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:22:37.368716 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de916a5-ef65-489d-86e6-b638abb7f841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7984ca6235ee49c6bc64239f528fb36e)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:22:37.370653 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de916a5-ef65-489d-86e6-b638abb7f841 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:22:37.417892 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5de916a5-ef65-489d-86e6-b638abb7f841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7984ca6235ee49c6bc64239f528fb36e', 'request_id': 'req-5de916a5-ef65-489d-86e6-b638abb7f841'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:22:37.423735 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5de916a5-ef65-489d-86e6-b638abb7f841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7984ca6235ee49c6bc64239f528fb36e', 'request_id': 'req-5de916a5-ef65-489d-86e6-b638abb7f841'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:22:37.429482 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5de916a5-ef65-489d-86e6-b638abb7f841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7984ca6235ee49c6bc64239f528fb36e', 'request_id': 'req-5de916a5-ef65-489d-86e6-b638abb7f841'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:22:37.441321 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5de916a5-ef65-489d-86e6-b638abb7f841 admin admin] The token cache is being invalidate because project 7984ca6235ee49c6bc64239f528fb36e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 23:22:37.441606 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5de916a5-ef65-489d-86e6-b638abb7f841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:22:37.442262 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5de916a5-ef65-489d-86e6-b638abb7f841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 23:22:37.443786 dv-cinder-840678-5-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1446/2895] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 9 23:22:37 2022] DELETE /identity/v3/projects/7984ca6235ee49c6bc64239f528fb36e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 23:25:36.754600 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb1191-1fd0-4ff4-be96-63da8bd09fcc admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 23:25:36.755976 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-54fb1191-1fd0-4ff4-be96-63da8bd09fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:25:36.887878 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb1191-1fd0-4ff4-be96-63da8bd09fcc admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 23:25:36.944272 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa02a3-bea6-4743-acf6-253e243d75e1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf9bc2fa959d4706b9f9e87509829059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0500e378ea374ca692e230bbd8781318', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 23:25:36.944913 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa02a3-bea6-4743-acf6-253e243d75e1 service placement] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 23:25:36.945197 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa02a3-bea6-4743-acf6-253e243d75e1 service placement] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 23:25:36.945476 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa02a3-bea6-4743-acf6-253e243d75e1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 23:25:36.947267 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4afa02a3-bea6-4743-acf6-253e243d75e1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 23:25:36.958671 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa02a3-bea6-4743-acf6-253e243d75e1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 23:25:36.960248 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa02a3-bea6-4743-acf6-253e243d75e1 service placement] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 23:25:36.969954 dv-cinder-840678-5-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1450/2896] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 9 23:25:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)